[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061138#comment-14061138
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
{quote}
It looks like your IDE did some strange things with the imports, and I'm not
sure why the toByteBuffer() line was added to WordCount since it won't compile
with it, but maybe that's just artifacts from patch against 2.0.0 and a rebase
will solve it.
{quote}
The import changes applies only to the trunk version not the 2.0-HEAD
As peer our discussion. I attached two files; 7200_v4_trunk has the trunk
changes that include: solution to the compilation problem in cql3 word count,
sleep fixes, log fix, and README addition.
7200_v4_2.0-HEAD has the 2.0-HEAD changes that include: sleep fixes, log fix,
and README addition.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt, 7200_v3.txt,
7200_v4_2.0-HEAD.txt, 7200_v4_trunk.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059064#comment-14059064
]
Brandon Williams commented on CASSANDRA-7200:
-
It looks like your IDE did some strange things with the imports, and I'm not
sure why the toByteBuffer() line was added to WordCount since it won't compile
with it, but maybe that's just artifacts from patch against 2.0.0 and a rebase
will solve it.
cql3 word count looks good, can we add similar logic about the sleep to word
count (and rebase aganst 2.0 head)?
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt, 7200_v3.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057868#comment-14057868
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
The latest NPE is already solved in
[CASSANDRA-6793|https://issues.apache.org/jira/browse/CASSANDRA-6793]
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057879#comment-14057879
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
The slowness is caused by using vnodes at the cassandra server which is not
recommended for the hadoop case. For instance, WordCountSetup relies on the
number of returned vnodes to determine the sleep time after creating the
keyspace which turned to be 256 seconds in current default installation of
Cassandra. I updated the README file to mention that disabaling vnodes is
recommended.
The Attached 7200_v2.txt includes the README update as well as the changes in
7200_v1.txt changes.
I also checked Cassandra 2.0. The hadoop_word_count example works fine, but the
cql3 word count has a bug that is already solved as I mentioned in the above
comment.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057904#comment-14057904
]
Brandon Williams commented on CASSANDRA-7200:
-
bq. I also checked Cassandra 2.0. The hadoop_word_count example works fine, but
the cql3 word count has a bug that is already solved as I mentioned in the
above comment.
That says it was fixed in 2.0.7, so I'm confused why we're encountering it.
bq. The slowness is caused by using vnodes at the cassandra server which is not
recommended for the hadoop case. For instance, WordCountSetup relies on the
number of returned vnodes to determine the sleep time after creating the
keyspace which turned to be 256 seconds in current default installation of
Cassandra.
Well, that's silly then, since it can both a) count actual nodes or b) poll for
actual schema agreement.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058038#comment-14058038
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
{quote}
That says it was fixed in 2.0.7, so I'm confused why we're encountering it.
{quote}
I used 2.0 for my test. 2.0.7 does not have the issue
{quote}
Well, that's silly then, since it can both a) count actual nodes or b) poll for
actual schema agreement.
{quote}
I used number of peers to get the actual nodes count. check 7200_v3.txt
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt, 7200_v3.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058041#comment-14058041
]
Brandon Williams commented on CASSANDRA-7200:
-
bq. I used 2.0 for my test. 2.0.7 does not have the issue
2.0-HEAD? That's the source of my confusion, we're on 2.0.9 now so we
shouldn't be seeing this, unless you mean 2.0.0.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt, 7200_v3.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058045#comment-14058045
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
{quote}
2.0-HEAD? That's the source of my confusion, we're on 2.0.9 now so we shouldn't
be seeing this, unless you mean 2.0.0.
{quote}
yes, I used 2.0.0. Sorry for not being clear.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt, 7200_v2.txt, 7200_v3.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055101#comment-14055101
]
Brandon Williams commented on CASSANDRA-7200:
-
A couple of things: word count is broken in 2.0 too, (quite likely for
different reasons though) so we need a version for that. Also,
word_count_setup never seems to exit, even though it is done and the box is
completely idle.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055108#comment-14055108
]
Brandon Williams commented on CASSANDRA-7200:
-
I take that last point back, it finally exited but seemed to spend a ridiculous
amount of time doing nothing.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055113#comment-14055113
]
Brandon Williams commented on CASSANDRA-7200:
-
looks like the cql3 word count has problems as well. Maybe for 2.1 we should
just simplify our lives and get rid of the old count and stick to cql3.
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055159#comment-14055159
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
I checked the 2.0 version of cql3 word count. it builds successfully and it
takes so long time to finish in both the old version and the cql3 version. is
that what you mean by broken?
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055188#comment-14055188
]
Brandon Williams commented on CASSANDRA-7200:
-
The slow idle period is a problem, but I think I only tested cql3 on 2.1
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055194#comment-14055194
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
word count of cql3 on 2.0 also fails with this exception when writing back to
cassandra in the reducer:
{code}
java.lang.NullPointerException
at
org.apache.cassandra.dht.Murmur3Partitioner.getToken(Murmur3Partitioner.java:89)
at
org.apache.cassandra.dht.Murmur3Partitioner.getToken(Murmur3Partitioner.java:39)
at org.apache.cassandra.client.RingCache.getRange(RingCache.java:117)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.write(CqlRecordWriter.java:162)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.write(CqlRecordWriter.java:62)
at
org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$ReducerToCassandra.reduce(Unknown Source)
at WordCount$ReducerToCassandra.reduce(Unknown Source)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
{code}
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.10
Attachments: 7200_v1.txt
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044072#comment-14044072
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
For the hadoop_cql3_word_count example:
word_count_setup is verified to work against the trunk . I could not reproduce
the problem.
The world_count also works correctly and produce the expected results. Running
word_count_counters after it generates the expected counts
cql3_worldcount.output_words.
The IOException above is a [known Hadoop
issue|https://issues.apache.org/jira/browse/HADOOP-2851] that's marked as won't
fix, and it can be safely ignored. it just logs an exception to the debug log,
despite no exception actually being thrown. When the log level is set to any
value other that DEBUG, the exception disappears.
the hadoop_word_count fails to build. Are we still interested in shipping it
with Cassandra?
Also, all of the examples shipped with Cassandra do not work on Windows. Are we
interested in supporting Windows examples?
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.9
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044082#comment-14044082
]
Brandon Williams commented on CASSANDRA-7200:
-
I think I discovered this against 2.0, not sure what the status is there now.
bq. The IOException above is a known Hadoop issue that's marked as won't fix,
and it can be safely ignored. it just logs an exception to the debug log,
despite no exception actually being thrown. When the log level is set to any
value other that DEBUG, the exception disappears.
We could silence it with log4j/logback configuration.
bq. the hadoop_word_count fails to build. Are we still interested in shipping
it with Cassandra?
Yes
bq. Also, all of the examples shipped with Cassandra do not work on Windows.
Are we interested in supporting Windows examples?
It's probably a good idea, but not critical. What breaks on windows?
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.9
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044085#comment-14044085
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
{quote}
It's probably a good idea, but not critical. What breaks on windows?
{quote}
These examples run by the shell scripts under bin directory. We need to provide
equivalent .bat files for Windows
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.9
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7200) word count broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044198#comment-14044198
]
Ala' Alkhaldi commented on CASSANDRA-7200:
--
{quot}
These examples run by the shell scripts under bin directory. We need to provide
equivalent .bat files for Windows
{quot}
Apparently, I was not accurate. Hadoop-core 1.0.3 does not support Windows
platform. The first release to support it is 2.1.0-beta ([see Hadoop release
notes|http://hadoop.apache.org/releases.html#25+August%2C+2013%3A+Release+2.1.0-beta+available]).
word count broken
-
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Assignee: Ala' Alkhaldi
Fix For: 2.0.9
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.init(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.init(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.init(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.init(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.init(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.init(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.init(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
