[jira] [Commented] (PHOENIX-3112) Partial row scan not handled correctly
[
https://issues.apache.org/jira/browse/PHOENIX-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15694378#comment-15694378
]
Guillaume salou commented on PHOENIX-3112:
--
We can't increased Increase default value of
hbase.client.scanner.max.result.size as much as we want, we still have missing
datas, and at the end java heap space.
Is it planned to fixed this bug ? In the past we have no problem with 4.6 /
Hbase 1.0 but since we have upgrade to 4.8.2/Hbase 1.2 this problem is blocker.
> Partial row scan not handled correctly
> --
>
> Key: PHOENIX-3112
> URL: https://issues.apache.org/jira/browse/PHOENIX-3112
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Pierre Lacave
>
> When doing a select of a relatively large table (a few touthands rows) some
> rows return partially missing.
> When increasing the fitler to return those specific rows, the values appear
> as expected
> {noformat}
> CREATE TABLE IF NOT EXISTS TEST (
> BUCKET VARCHAR,
> TIMESTAMP_DATE TIMESTAMP,
> TIMESTAMP UNSIGNED_LONG NOT NULL,
> SRC VARCHAR,
> DST VARCHAR,
> ID VARCHAR,
> ION VARCHAR,
> IC BOOLEAN NOT NULL,
> MI UNSIGNED_LONG,
> AV UNSIGNED_LONG,
> MA UNSIGNED_LONG,
> CNT UNSIGNED_LONG,
> DUMMY VARCHAR
> CONSTRAINT pk PRIMARY KEY (BUCKET, TIMESTAMP DESC, SRC, DST, ID, ION, IC)
> );{noformat}
> using a python script to generate a CSV with 5000 rows
> {noformat}
> for i in xrange(5000):
> print "5SEC,2016-07-21
> 07:25:35.{i},146908593500{i},,AAA,,,false,{i}1181000,1788000{i},2497001{i},{i},a{i}".format(i=i)
> {noformat}
> bulk inserting the csv in the table
> {noformat}
> phoenix/bin/psql.py localhost -t TEST large.csv
> {noformat}
> here we can see one row that contains no TIMESTAMP_DATE and null values in MI
> and MA
> {noformat}
> 0: jdbc:phoenix:localhost:2181> select * from TEST
>
> +-+--+---+---+--+---+---++--+--+--+---++
> | BUCKET | TIMESTAMP_DATE | TIMESTAMP |SRC| DST |
> ID |ION| IC | MI | AV | MA |
> CNT | DUMMY
> |
> +-+--+---+---+--+---+---++--+--+--+---++
> | 5SEC| 2016-07-21 07:25:35.100 | 1469085935001000 | | AAA |
> | | false | 10001181000 | 17880001000 | 24970011000 |
> 1000 |
> a1000 |
> | 5SEC| 2016-07-21 07:25:35.999 | 146908593500999 | | AAA |
> | | false | 9991181000 | 1788000999 | 2497001999 | 999
> | a999
> |
> | 5SEC| 2016-07-21 07:25:35.998 | 146908593500998 | | AAA |
> | | false | 9981181000 | 1788000998 | 2497001998 | 998
> | a998
> |
> | 5SEC| | 146908593500997 | | AAA |
> | | false | null | 1788000997 | null | 997
> |
> |
> | 5SEC| 2016-07-21 07:25:35.996 | 146908593500996 | | AAA |
> | | false | 9961181000 | 1788000996 | 2497001996 | 996
> | a996
> |
> | 5SEC| 2016-07-21 07:25:35.995 | 146908593500995 | | AAA |
> | | false | 9951181000 | 1788000995 | 2497001995 | 995
> | a995
> |
> | 5SEC| 2016-07-21 07:25:35.994 | 146908593500994 | | AAA |
> | | false | 9941181000 | 1788000994 | 2497001994 | 994
> | a994
> |
>
> {noformat}
> but when selecting that row specifically the values are correct
> {noformat}
> 0: jdbc:phoenix:localhost:2181> select * from TEST where timestamp =
> 146908593500997;
> +-+--+--+---+--+---+---++-+-
[jira] [Commented] (PHOENIX-2298) Problem storing with pig on a salted table
[ https://issues.apache.org/jira/browse/PHOENIX-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939863#comment-14939863 ] Guillaume salou commented on PHOENIX-2298: -- Thanks, I am using 4.3.0-1 > Problem storing with pig on a salted table > -- > > Key: PHOENIX-2298 > URL: https://issues.apache.org/jira/browse/PHOENIX-2298 > Project: Phoenix > Issue Type: Bug >Reporter: Guillaume salou > > When I try to upsert via pigStorage on a salted table I get this error. > Store ... using org.apache.phoenix.pig.PhoenixHBaseStorage(); > first field of the table : > CurrentTime() asINTERNALTS:datetime, > This date is not used in the primary key of the table. > Works perfectly on a non salted table. > Caused by: java.lang.RuntimeException: Unable to process column _SALT:BINARY, > innerMessage=org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. BINARY cannot be coerced to DATE > at > org.apache.phoenix.pig.writable.PhoenixPigDBWritable.write(PhoenixPigDBWritable.java:66) > at > org.apache.phoenix.mapreduce.PhoenixRecordWriter.write(PhoenixRecordWriter.java:78) > at > org.apache.phoenix.mapreduce.PhoenixRecordWriter.write(PhoenixRecordWriter.java:39) > at > org.apache.phoenix.pig.PhoenixHBaseStorage.putNext(PhoenixHBaseStorage.java:182) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > at > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558) > at > org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85) > at > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) > at > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. BINARY cannot be coerced to DATE > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > at org.apache.phoenix.schema.types.PDate.toObject(PDate.java:77) > at > org.apache.phoenix.pig.util.TypeUtil.castPigTypeToPhoenix(TypeUtil.java:208) > at > org.apache.phoenix.pig.writable.PhoenixPigDBWritable.convertTypeSpecificValue(PhoenixPigDBWritable.java:79) > at > org.apache.phoenix.pig.writable.PhoenixPigDBWritable.write(PhoenixPigDBWritable.java:59) > ... 21 more > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. BINARY cannot be coerced to DATE > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:68) > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:133) > ... 26 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2298) Problem storing with pig on a salted table
Guillaume salou created PHOENIX-2298: Summary: Problem storing with pig on a salted table Key: PHOENIX-2298 URL: https://issues.apache.org/jira/browse/PHOENIX-2298 Project: Phoenix Issue Type: Bug Reporter: Guillaume salou When I try to upsert via pigStorage on a salted table I get this error. Store ... using org.apache.phoenix.pig.PhoenixHBaseStorage(); first field of the table : CurrentTime() asINTERNALTS:datetime, This date is not used in the primary key of the table. Works perfectly on a non salted table. Caused by: java.lang.RuntimeException: Unable to process column _SALT:BINARY, innerMessage=org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. BINARY cannot be coerced to DATE at org.apache.phoenix.pig.writable.PhoenixPigDBWritable.write(PhoenixPigDBWritable.java:66) at org.apache.phoenix.mapreduce.PhoenixRecordWriter.write(PhoenixRecordWriter.java:78) at org.apache.phoenix.mapreduce.PhoenixRecordWriter.write(PhoenixRecordWriter.java:39) at org.apache.phoenix.pig.PhoenixHBaseStorage.putNext(PhoenixHBaseStorage.java:182) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.phoenix.schema.ConstraintViolationException: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. BINARY cannot be coerced to DATE at org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) at org.apache.phoenix.schema.types.PDate.toObject(PDate.java:77) at org.apache.phoenix.pig.util.TypeUtil.castPigTypeToPhoenix(TypeUtil.java:208) at org.apache.phoenix.pig.writable.PhoenixPigDBWritable.convertTypeSpecificValue(PhoenixPigDBWritable.java:79) at org.apache.phoenix.pig.writable.PhoenixPigDBWritable.write(PhoenixPigDBWritable.java:59) ... 21 more Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. BINARY cannot be coerced to DATE at org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:68) at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:133) ... 26 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
