[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705457#comment-14705457 ] Jeremiah Jordan commented on CASSANDRA-7410: ping > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712687#comment-14712687 ] Piotr Kołaczkowski commented on CASSANDRA-7410: --- +1 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745593#comment-14745593 ] Aleksey Yeschenko commented on CASSANDRA-7410: -- Committed to cassandra-2.1 as [c7b40735789c840529002eb3c11d8731f460d61c|https://github.com/apache/cassandra/commit/c7b40735789c840529002eb3c11d8731f460d61c] and merged upstream. 3.0 merge went without conflicts, but I did not verify if it works or not. If something is broken there, it's gonna be on Alex and Piotr. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.1.x, 2.2.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, CASSANDRA-7410-v1-2.2.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096201#comment-14096201 ] Brandon Williams commented on CASSANDRA-7410: - Can you rebase? Doesn't quite work on 2.0 with the latest CASSANDRA-6927 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.10, 2.1.1 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036614#comment-14036614
]
Alex Liu commented on CASSANDRA-7410:
-
The patch is on top of CASSANDRA-6927. The following url parameters are added
to CqlStorage
{code}
[&bulk_output_format=true|false]
[&bulk_cf_schema=bulk_cf_schema]
[&bulk_insert_statement=bulk_insert_statement]
[&bulk_output_location=]
[&bulk_buff_size=]
[&bulk_stream_throttle=]
[&bulk_max_failed_hosts=]
{code}
Note: legacy thrift based BulkOutputFormat is not supported.
> Pig support for BulkOutputFormat as a parameter in url
> --
>
> Key: CASSANDRA-7410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 2.0.9
>
> Attachments: 7410-2.0-branch.txt
>
>
> Add BulkOutputFormat support in Pig url
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347505#comment-14347505
]
Alex Liu commented on CASSANDRA-7410:
-
{{CqlBulkRecordWriter}} comes with its own {{ExternalClient}} which uses a
wrong approach for generating CFMetaData (and cfId). We can fix the issue by
using the approach found in {{BulkLoader.ExternalClient}} which fetches
CFMetadata from the cluster using CQL.
> Pig support for BulkOutputFormat as a parameter in url
> --
>
> Key: CASSANDRA-7410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 2.0.13
>
> Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt,
> 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt
>
>
> Add BulkOutputFormat support in Pig url
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132792#comment-14132792 ] Brandon Williams commented on CASSANDRA-7410: - Ping [~alexliu68] > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135690#comment-14135690 ] Alex Liu commented on CASSANDRA-7410: - checking > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138154#comment-14138154 ] Brandon Williams commented on CASSANDRA-7410: - I'd prefer just rebasing the first much smaller patch, rather than do such a large refactoring in a minor release. We can do that in 2.1 if that's what we want. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt, 7410-v2-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284045#comment-14284045 ] Brandon Williams commented on CASSANDRA-7410: - Sorry, missed this one because it wasn't marked PA. Can you rebase and also push a 2.1 version? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.13 > > Attachments: 7410-2.0-branch.txt, 7410-v2-2.0-branch.txt, > 7410-v3-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284918#comment-14284918
]
Alex Liu commented on CASSANDRA-7410:
-
I got following sstable loading error after bulk writing.
{code}
DEBUG [Thread-193] 2015-01-20 16:43:53,751 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-TOC.txt
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-TOC.txt
DEBUG [Thread-193] 2015-01-20 16:43:53,751 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Statistics.db
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Statistics.db
DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Digest.sha1
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Digest.sha1
DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Filter.db
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Filter.db
DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Index.db
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Index.db
DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-CompressionInfo.db
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-CompressionInfo.db
DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Data.db
to
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Data.db
DEBUG [Thrift:9] 2015-01-20 16:43:53,758 computing ranges for
8e56ee4c6ef2c35b3d97c6abeefe8b92
DEBUG [Thread-191] 2015-01-20 16:43:53,786 Load metadata for
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1
INFO [Thread-191] 2015-01-20 16:43:53,786 Opening
/var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1
(52 bytes)
INFO [Thread-191] 2015-01-20 16:43:53,802 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Executing streaming plan for Bulk Load
INFO [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,802 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Starting streaming to /127.0.0.1
DEBUG [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,802 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Sending stream init for incoming stream
DEBUG [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,805 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Sending stream init for outgoing stream
DEBUG [ACCEPT-/127.0.0.1] 2015-01-20 16:43:53,805 Connection version 2 from
/127.0.0.1
DEBUG [ACCEPT-/127.0.0.1] 2015-01-20 16:43:53,806 Connection version 2 from
/127.0.0.1
INFO [STREAM-INIT-/127.0.0.1:52052] 2015-01-20 16:43:53,806 [Stream
#931af290-a106-11e4-ae05-d11cec192498 ID#0] Creating new streaming plan for
Bulk Load
DEBUG [STREAM-OUT-/127.0.0.1] 2015-01-20 16:43:53,806 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Sending Prepare (0 requests, 1 files}
INFO [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,807 [Stream
#931af290-a106-11e4-ae05-d11cec192498 ID#0] Prepare completed. Receiving 0
files(0 bytes), sending 1 files(52 bytes)
INFO [STREAM-INIT-/127.0.0.1:52052] 2015-01-20 16:43:53,807 [Stream
#931af290-a106-11e4-ae05-d11cec192498, ID#0] Received streaming plan for Bulk
Load
INFO [STREAM-INIT-/127.0.0.1:52053] 2015-01-20 16:43:53,808 [Stream
#931af290-a106-11e4-ae05-d11cec192498, ID#0] Received streaming plan for Bulk
Load
DEBUG [STREAM-IN-/127.0.0.1] 2015-01-20 16:43:53,808 [Stream
#931af290-a106-11e4-ae05-d11cec192498] Received Prepare (0 requests, 1 files}
INFO [STREAM-IN-/127.0.0.1] 2015-01-20 16:43:53,808 [Stream
#931af290-a106-11e4-ae05-d11cec192498 ID#0] Prepare completed. Receiving 1
files(52 bytes), sending 0 files(0 byt
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365884#comment-14365884 ] Alex Liu commented on CASSANDRA-7410: - Waiting for CASSANDRA-8358 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365910#comment-14365910 ] Alex Liu commented on CASSANDRA-7410: - [~brandon.williams] Do you have time to review it? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365925#comment-14365925 ] Brandon Williams commented on CASSANDRA-7410: - Seems like waiting on CASSANDRA-8358 is the right move, at least for trunk, so we don't compound a problem. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484096#comment-14484096 ] Aleksey Yeschenko commented on CASSANDRA-7410: -- CASSANDRA-8358 is taking a bit longer than I expected to review/commit. Could be delayed by a week or so more. Can you guys go ahead and review/commit this without 8358? I'll rebase CASSANDRA-8358 afterwards. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496547#comment-14496547
]
Piotr Kołaczkowski commented on CASSANDRA-7410:
---
org/apache/cassandra/hadoop/pig/CqlNativeStorage.java:149
{noformat}
if (t.getType(0) == DataType.TUPLE)
{
if (bulkOutputFormat)
{
cqlQueryFromTuple(null, t, 0);
}
else if (t.getType(1) == DataType.TUPLE)
{
Map key = tupleToKeyMap((Tuple)t.get(0));
cqlQueryFromTuple(key, t, 1);
}
else
throw new IOException("Second argument in output must be a
tuple");
}
else
throw new IOException("First argument in output must be a tuple");
{noformat}
Personally, I don't like this input validation style.
Much better to validate input in a flat way at the beginning:
{noformat}
if (t.getType(0) != DataType.TUPLE)
throw
if (t.getType(1) != DataType.TUPLE)
throw
// now we know input is ok, so we can focus on doing real stuff
{noformat}
Moreover, {{cqlQueryFromTuple}} does the same validation again...
org.apache.cassandra.hadoop.pig.CqlNativeStorage#setStoreLocation:
This method is a copy-paste from
org.apache.cassandra.hadoop.pig.CqlStorage#setStoreLocation
with only a minor section related to bulkOutputFormat added.
Any reason for not using super.setStoreLocation()?
org/apache/cassandra/io/sstable/CQLSSTableWriter.java:
{noformat}
try
{
Schema.instance.load(ksm);
}
catch (Exception e)
{
//It may get an exception of Attempting to load already
loaded column family
}
{noformat}
Ok, I get it, but what if it tries to load it for the first time and fails? It
doesn't even inform the user that something bad happened and why.
Also, can you elaborate more on why it may want to load it multiple times?
> Pig support for BulkOutputFormat as a parameter in url
> --
>
> Key: CASSANDRA-7410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 2.0.15
>
> Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt,
> 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt,
> CASSANDRA-7410-v2-2.1-branch.txt
>
>
> Add BulkOutputFormat support in Pig url
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498218#comment-14498218 ] Alex Liu commented on CASSANDRA-7410: - Any reason for not using super.setStoreLocation()? -- We will remove CqlStorage class soon, so try to not couple with it any more. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498768#comment-14498768 ] Alex Liu commented on CASSANDRA-7410: - This change is no longer need, because I port the patch on 2.1 back to 2.0 which uses BulkLoader.ExternalClient . > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503144#comment-14503144
]
Piotr Kołaczkowski commented on CASSANDRA-7410:
---
Patch doesn't apply to 2.0 branch:
{noformat}
pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git fetch
pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git checkout cassandra-2.0
Already on 'cassandra-2.0'
Your branch is up-to-date with 'origin/cassandra-2.0'.
pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git apply 7410-v3-2.0-branch.txt
7410-v3-2.0-branch.txt:195: trailing whitespace.
"[columns=][where_clause=]" +
error: patch failed:
src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java:345
error: src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java: patch
does not apply
{noformat}
> Pig support for BulkOutputFormat as a parameter in url
> --
>
> Key: CASSANDRA-7410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 2.0.15
>
> Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt,
> 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt,
> CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt,
> CASSANDRA-7410-v4-2.0-branch.txt
>
>
> Add BulkOutputFormat support in Pig url
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503183#comment-14503183
]
Piotr Kołaczkowski commented on CASSANDRA-7410:
---
org/apache/cassandra/hadoop/pig/CqlNativeStorage.java:342
{noformat}
private boolean serverEntryped()
{
if (!StringUtils.isEmpty(internodeEncrypt))
return InternodeEncryption.none !=
InternodeEncryption.valueOf(internodeEncrypt.toLowerCase());
return false;
}
{noformat}
Typo: serverEntryped -> serverEncrypted.
Also, the if can be slightly simplified:
{noformat}
private boolean serverEntryped()
{
return !StringUtils.isEmpty(internodeEncrypt) &&
InternodeEncryption.none !=
InternodeEncryption.valueOf(internodeEncrypt.toLowerCase());
}
{noformat}
Other than this, it looks good.
+1
> Pig support for BulkOutputFormat as a parameter in url
> --
>
> Key: CASSANDRA-7410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 2.0.15
>
> Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt,
> 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt,
> CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt,
> CASSANDRA-7410-v4-2.0-branch.txt
>
>
> Add BulkOutputFormat support in Pig url
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
