[jira] [Created] (CARBONDATA-1026) dependency commons-lang3 conflict whith spark2.1
yangxy created CARBONDATA-1026: -- Summary: dependency commons-lang3 conflict whith spark2.1 Key: CARBONDATA-1026 URL: https://issues.apache.org/jira/browse/CARBONDATA-1026 Project: CarbonData Issue Type: Bug Reporter: yangxy 17/05/05 13:58:41 ERROR scheduler.TaskSetManager: Task 0 in stage 1.0 failed 4 times; aborting job org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4, node03, executor 3): java.io.InvalidClassException: org.apache.commons.lang3.time.FastDateParser; local class incompatible: stream classdesc serialVersionUID = 3, local class serialVersionUID = 2 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1025) dependency univocity-parsers conflict whith spark2.1
yangxy created CARBONDATA-1025: -- Summary: dependency univocity-parsers conflict whith spark2.1 Key: CARBONDATA-1025 URL: https://issues.apache.org/jira/browse/CARBONDATA-1025 Project: CarbonData Issue Type: Bug Reporter: yangxy Caused by: java.lang.NoSuchMethodError: com.univocity.parsers.csv.CsvParserSettings.setUnescapedQuoteHandling(Lcom/univocity/parsers/csv/UnescapedQuoteHandling;)V at org.apache.spark.sql.execution.datasources.csv.CsvReader.(CSVParser.scala:47) at org.apache.spark.sql.execution.datasources.csv.CSVFileFormat$$anonfun$buildReader$1.apply(CSVFileFormat.scala:158) at org.apache.spark.sql.execution.datasources.csv.CSVFileFormat$$anonfun$buildReader$1.apply(CSVFileFormat.scala:146) at org.apache.spark.sql.execution.datasources.FileFormat$$anon$1.apply(FileFormat.scala:138) at org.apache.spark.sql.execution.datasources.FileFormat$$anon$1.apply(FileFormat.scala:122) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:168) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:109) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithoutKey$(Unknown Source) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377) at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1024) Support reading Float Data Type In Hive From Carbon Table
anubhav tarar created CARBONDATA-1024: - Summary: Support reading Float Data Type In Hive From Carbon Table Key: CARBONDATA-1024 URL: https://issues.apache.org/jira/browse/CARBONDATA-1024 Project: CarbonData Issue Type: Improvement Components: hive-integration Environment: hive 1.2,spark 2,1 Reporter: anubhav tarar Assignee: anubhav tarar Priority: Minor currently float data type is not supported in hive,can we support it? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1023) Able to do load from dataframe with byte data type in carbon table
anubhav tarar created CARBONDATA-1023:
-
Summary: Able to do load from dataframe with byte data type in
carbon table
Key: CARBONDATA-1023
URL: https://issues.apache.org/jira/browse/CARBONDATA-1023
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.0.0-incubating
Environment: spark common
Reporter: anubhav tarar
Assignee: anubhav tarar
Priority: Trivial
i am able to load data with bytetype from dataframe in a carbon table
as far as documentation says byte type is not supported
val rdd1 = sqlContext.sparkContext.parallelize(
Row("byte",1234.toByte) :: Nil)
val schema1 = StructType(
StructField("string",StringType, nullable = false) ::
StructField("byte",ByteType, nullable = false) :: Nil)
val dataFrame = sqlContext.createDataFrame(rdd1, schema1)
dataFrame.write
.format("carbondata")
.option("tableName", "carbon1")
.option("compress", "true")
.mode(SaveMode.Overwrite)
.save()
sql("select * from carbon1").show()
+--++
|string|byte|
+--++
| byte| -46|
+--++
if we try to create a table with byte type it shows error
sql(
"CREATE TABLE restructure (empno byte, empname String, designation
String, doj Timestamp, " +
"workgroupcategory int, workgroupcategoryname String, deptno int,
deptname String, " +
"projectcode int, projectjoindate Timestamp, projectenddate
Timestamp,attendance int," +
"utilization int,salary int) STORED BY 'org.apache.carbondata.format'")
result
org.apache.carbondata.spark.exception.MalformedCarbonCommandException:
Unsupported data type: StructField(empno,ByteType,true).getType
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CARBONDATA-1022) Get errors while do "select" query in Spark-shell
Liang Chen created CARBONDATA-1022: -- Summary: Get errors while do "select" query in Spark-shell Key: CARBONDATA-1022 URL: https://issues.apache.org/jira/browse/CARBONDATA-1022 Project: CarbonData Issue Type: Bug Components: sql Reporter: Liang Chen -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1021) Update compact for code style and unne
chenerlu created CARBONDATA-1021: Summary: Update compact for code style and unne Key: CARBONDATA-1021 URL: https://issues.apache.org/jira/browse/CARBONDATA-1021 Project: CarbonData Issue Type: Improvement Reporter: chenerlu -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1020) Update project info from incubator to top level for graduation
Liang Chen created CARBONDATA-1020: -- Summary: Update project info from incubator to top level for graduation Key: CARBONDATA-1020 URL: https://issues.apache.org/jira/browse/CARBONDATA-1020 Project: CarbonData Issue Type: Task Reporter: Liang Chen Assignee: Liang Chen -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CARBONDATA-1019) Like Filter Pushdown
sounak chakraborty created CARBONDATA-1019: -- Summary: Like Filter Pushdown Key: CARBONDATA-1019 URL: https://issues.apache.org/jira/browse/CARBONDATA-1019 Project: CarbonData Issue Type: Bug Reporter: sounak chakraborty Assignee: sounak chakraborty Like Filter Pushdown in carbon layer to convert into Range Filter. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
