Re: python client example
Thanks guys, playing around with pycassa right now. seems pretty good. On Thu, Jan 13, 2011 at 2:56 PM, Aaron Morton wrote: > Ah, i get it now. The python code generated from running ant gen-thrift-py > . > > IMHO Start with Pycassa *even* if you want to go your own way later. It > solves a lot of problems for you and will save you time. > > A > > > On 14 Jan, 2011,at 11:46 AM, Tyler Hobbs wrote: > > Right, python-cassandra just provides the raw Thrift API, which is no fun > at all. You should start out with pycassa. > > - Tyler > > On Thu, Jan 13, 2011 at 4:45 PM, Aaron Morton wrote: > >> Sorry, I meant where did you get python-cassandra from on the web. >> >> Can you use Pycassa, even just as a learning experience ? There is a >> tutorial here http://pycassa.github.com/pycassa/tutorial.html >> >> <http://pycassa.github.com/pycassa/tutorial.html>A >> >> >> >> On 14 Jan, 2011,at 11:42 AM, felix gao wrote: >> >> this is where it is stored >> >> /opt/local/Library/Frameworks/Python.framework/Versions/26/lib/python2.6/site-packages/ >> >> >> On Thu, Jan 13, 2011 at 2:39 PM, Aaron Morton wrote: >> >>> Pycassa https://github.com/pycassa/pycassa >>> Has documentation here http://pycassa.github.com/pycassa/ >>> >>> >>> <https://github.com/pycassa/pycassa>Where does python-cassandra live ? >>> Aaron >>> >>> >>> On 14 Jan, 2011,at 11:34 AM, felix gao wrote: >>> >>> Guys, >>> >>> I just installed python-cassandra 0.6.1 and Thrift 0.5.0 on my machine >>> and I would like to query against also write into a cassandra server. I >>> guess i am pretty weak in google-fu, there isn't any examples for me get >>> started with. Please help me on how to do this. >>> >>> Thanks, >>> >>> Felix >>> >>> >> >
Re: python client example
this is where it is stored /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/ On Thu, Jan 13, 2011 at 2:39 PM, Aaron Morton wrote: > Pycassa https://github.com/pycassa/pycassa > Has documentation here http://pycassa.github.com/pycassa/ > > <https://github.com/pycassa/pycassa>Where does python-cassandra live ? > Aaron > > On 14 Jan, 2011,at 11:34 AM, felix gao wrote: > > Guys, > > I just installed python-cassandra 0.6.1 and Thrift 0.5.0 on my machine and > I would like to query against also write into a cassandra server. I guess i > am pretty weak in google-fu, there isn't any examples for me get started > with. Please help me on how to do this. > > Thanks, > > Felix > >
python client example
Guys, I just installed python-cassandra 0.6.1 and Thrift 0.5.0 on my machine and I would like to query against also write into a cassandra server. I guess i am pretty weak in google-fu, there isn't any examples for me get started with. Please help me on how to do this. Thanks, Felix
cassandra 0.7.0 noob question
Hi all, I started cassandra with very thing untouched in the conf folder, when I examine the cassandra.yaml file, there seems to be a default keyspace defined like below. keyspaces: - name: Keyspace1 replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy replication_factor: 1 column_families: - name: Standard1 my question is when I ran the cassandra-cli and show keyspaces; only system keyspace is there. What is going on? Thanks, Felix
Re: pig cassandra contribution
Ignore the above error, I somehow passed that stage. However, I am still having problem with it. grunt> register /home/felix/pig-0.7.0/pig-0.7.1-dev.jar; register /home/felix/cassandra/lib/libthrift.jar; grunt> rows = LOAD 'cassandra://test/data' USING CassandraStorage(); grunt> cols = FOREACH rows GENERATE flatten($1); grunt> colnames = FOREACH cols GENERATE $0; grunt> limit_colnames = limit colnames 10; grunt> dump limit_colnames 2011-01-05 15:44:17,378 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId= 2011-01-05 15:44:17,460 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: Store(file:/tmp/temp-1545399343/tmp576746049:org.apache.pig.builtin.BinStorage) - 1-27 Operator Key: 1-27) 2011-01-05 15:44:17,507 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1 2011-01-05 15:44:17,507 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 2011-01-05 15:44:17,533 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:17,539 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:17,539 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 2011-01-05 15:44:21,785 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job 2011-01-05 15:44:21,841 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:21,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. 2011-01-05 15:44:21,846 [Thread-5] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2011-01-05 15:44:22,115 [Thread-5] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:22,133 [Thread-5] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:22,344 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 2011-01-05 15:44:22,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2117: Unexpected error when launching map reduce job. Details at logfile: /home/felix/cassandra/contrib/pig/pig_1294263823129.log cat pig_1294263823129.log Pig Stack Trace --- ERROR 2117: Unexpected error when launching map reduce job. org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias limit_colnames at org.apache.pig.PigServer.openIterator(PigServer.java:521) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) at org.apache.pig.Main.main(Main.java:357) Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias limit_colnames at org.apache.pig.PigServer.store(PigServer.java:577) at org.apache.pig.PigServer.openIterator(PigServer.java:504) ... 6 more Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2117: Unexpected error when launching map reduce job. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:209) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:835) at org.apache.pig.PigServer.store(PigServer.java:569) ... 7 more Caused by: java.lang.RuntimeException: Could not resolve error that occured when launching map reduce job: java.lang.ExceptionInInitializerError at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandler.uncaughtException(MapReduceLauncher.java:510) at java.lang.Thread.dispatchUncaughtException(Thread.java:1831) On Wed, Jan 5, 2011 at 12:02 PM, felix gao wrote: >
pig cassandra contribution
I am having problem running the cassandra_loadfunc.jar on my build of cassandra. PIG_CLASSPATH=:bin/../build/cassandra_loadfunc.jar::bin/../../..//lib/antlr-3.1.3.jar:bin/../../..//lib/avro-1.2.0-dev.jar:bin/../../..//lib/clhm-production.jar:bin/../../..//lib/commons-cli-1.1.jar:bin/../../..//lib/commons-codec-1.2.jar:bin/../../..//lib/commons-collections-3.2.1.jar:bin/../../..//lib/commons-lang-2.4.jar:bin/../../..//lib/google-collections-1.0.jar:bin/../../..//lib/hadoop-core-0.20.1.jar:bin/../../..//lib/high-scale-lib.jar:bin/../../..//lib/jackson-core-asl-1.4.0.jar:bin/../../..//lib/jackson-mapper-asl-1.4.0.jar:bin/../../..//lib/jline-0.9.94.jar:bin/../../..//lib/json-simple-1.1.jar:bin/../../..//lib/libthrift.jar:bin/../../..//lib/log4j-1.2.14.jar:bin/../../..//lib/slf4j-api-1.5.8.jar:bin/../../..//lib/slf4j-log4j12-1.5.8.jar:bin/../../..//lib/spymemcached-2.4.2.jar:bin/../../..//lib/zapcat-1.2.jar:bin/../../..//build/lib/jars/ant-1.6.5.jar:bin/../../..//build/lib/jars/apache-rat-0.6.jar:bin/../../..//build/lib/jars/apache-rat-core-0.6.jar:bin/../../..//build/lib/jars/apache-rat-tasks-0.6.jar:bin/../../..//build/lib/jars/asm-3.2.jar:bin/../../..//build/lib/jars/avalon-framework-4.1.3.jar:bin/../../..//build/lib/jars/commons-cli-1.1.jar:bin/../../..//build/lib/jars/commons-collections-3.2.jar:bin/../../..//build/lib/jars/commons-lang-2.1.jar:bin/../../..//build/lib/jars/commons-logging-1.1.1.jar:bin/../../..//build/lib/jars/junit-4.6.jar:bin/../../..//build/lib/jars/log4j-1.2.12.jar:bin/../../..//build/lib/jars/logkit-1.0.1.jar:bin/../../..//build/lib/jars/paranamer-ant-2.1.jar:bin/../../..//build/lib/jars/paranamer-generator-2.1.jar:bin/../../..//build/lib/jars/qdox-1.10.jar:bin/../../..//build/lib/jars/servlet-api-2.3.jar:bin/../../..//build/apache-cassandra-0.6.4.jar:bin/../../..//build/ivy-2.1.0.jar:/usr/local/pig-0.7.0/pig.jar In Grunt I did register again just in case it is not picked up by the classpath register /usr/local/pig-0.7.0/pig.jar; register /home/felix/cassandra/lib/libthrift.jar; register /home/felix/cassandra/contrib/pig/build/cassandra_loadfunc.jar grunt> rows = LOAD 'cassandra://test.data' USING CassandraStorge(); 2011-01-05 13:50:50,071 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorge using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.] Details at logfile: /home/felix/cassandra/contrib/pig/pig_1294257032719.log the log file contains Pig Stack Trace --- ERROR 1070: Could not resolve CassandraStorge using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.] java.lang.RuntimeException: Cannot instantiate:CassandraStorge at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:455) at org.apache.pig.impl.logicalLayer.parser.QueryParser.NonEvalFuncSpec(QueryParser.java:5087) at org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1434) at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1245) at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:911) at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:700) at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63) at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114) at org.apache.pig.PigServer.registerQuery(PigServer.java:425) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) at org.apache.pig.Main.main(Main.java:357) Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve CassandraStorge using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.] at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:440) at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:452) ... 15 more Running hadoop 0.20.2 with pig0.7.0 and have to use cassandra 0.6.4. Thanks, Felix
