Build failed in Hudson: Hive-trunk-h0.17 #5
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/5/changes
--
[...truncated 16559 lines...]
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/build/ql/test/logs/negative/unknown_column2.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/src/test/results/compiler/errors/unknown_column2.q.out
[junit] Done query: unknown_column2.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150522_1308293018.txt
[junit] Begin query: unknown_column3.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/build/ql/test/logs/negative/unknown_column3.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/src/test/results/compiler/errors/unknown_column3.q.out
[junit] Done query: unknown_column3.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150522_132389553.txt
[junit] Begin query: unknown_column4.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/build/ql/test/logs/negative/unknown_column4.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/src/test/results/compiler/errors/unknown_column4.q.out
[junit] Done query: unknown_column4.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150522_241555180.txt
[junit] Begin query: unknown_column5.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/build/ql/test/logs/negative/unknown_column5.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/src/test/results/compiler/errors/unknown_column5.q.out
[junit] Done query: unknown_column5.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150522_-1645557578.txt
[junit] Begin query: unknown_column6.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/build/ql/test/logs/negative/unknown_column6.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/src/test/results/compiler/errors/unknown_column6.q.out
[junit] Done query: unknown_column6.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150522_439843309.txt
[junit] Begin query: unknown_function1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition
Build failed in Hudson: Hive-trunk-h0.18 #6
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/6/changes
--
[...truncated 18998 lines...]
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column2.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column2.q.out
[junit] Done query: unknown_column2.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150624_-1473799596.txt
[junit] Begin query: unknown_column3.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column3.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column3.q.out
[junit] Done query: unknown_column3.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150624_836629494.txt
[junit] Begin query: unknown_column4.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column4.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column4.q.out
[junit] Done query: unknown_column4.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150624_1364752236.txt
[junit] Begin query: unknown_column5.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column5.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column5.q.out
[junit] Done query: unknown_column5.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150624_-1520284702.txt
[junit] Begin query: unknown_column6.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column6.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column6.q.out
[junit] Done query: unknown_column6.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150624_548713664.txt
[junit] Begin query: unknown_function1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart
Build failed in Hudson: Hive-trunk-h0.19 #5
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/5/changes
--
[...truncated 18617 lines...]
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column2.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column2.q.out
[junit] Done query: unknown_column2.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150723_-1534377423.txt
[junit] Begin query: unknown_column3.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column3.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column3.q.out
[junit] Done query: unknown_column3.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150723_1980966805.txt
[junit] Begin query: unknown_column4.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column4.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column4.q.out
[junit] Done query: unknown_column4.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150723_1127360653.txt
[junit] Begin query: unknown_column5.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column5.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column5.q.out
[junit] Done query: unknown_column5.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150723_-677690512.txt
[junit] Begin query: unknown_column6.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column6.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column6.q.out
[junit] Done query: unknown_column6.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902150724_60150161.txt
[junit] Begin query: unknown_function1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition
[jira] Created: (HIVE-290) Add back generator for complex.seq
Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-290) Add back generator for complex.seq
[ https://issues.apache.org/jira/browse/HIVE-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-290: Component/s: Serializers/Deserializers Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-290) Add back generator for complex.seq
[ https://issues.apache.org/jira/browse/HIVE-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-290: Attachment: HIVE-290.1.patch Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao Attachments: HIVE-290.1.patch A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-290) Add back generator for complex.seq
[ https://issues.apache.org/jira/browse/HIVE-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-290: Attachment: complex.seq New complex.seq uses BytesWritable as key instead of ByteWritable to make sure it can be read without Hive. (ByteWritable is a WritableClass in Hive) Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao Attachments: complex.seq, HIVE-290.1.patch A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-291) [Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful
[Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful -- Key: HIVE-291 URL: https://issues.apache.org/jira/browse/HIVE-291 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful. If map-side aggregation is not reducing the number of output rows, it is a drain on the mapper, since it is consuming memory and performing unnecessary hash lookups -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-290) Add back generator for complex.seq
[ https://issues.apache.org/jira/browse/HIVE-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12673770#action_12673770 ] Namit Jain commented on HIVE-290: - something doesn't seem right - the first patch contains a bunch of files, whereas the second only the binary. shouldn't the files in the first patch also be included ? Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao Attachments: complex.seq, HIVE-290.1.patch A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-291) [Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful
[ https://issues.apache.org/jira/browse/HIVE-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-291: Attachment: 291.1.txt [Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful -- Key: HIVE-291 URL: https://issues.apache.org/jira/browse/HIVE-291 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: 291.1.txt Map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful. If map-side aggregation is not reducing the number of output rows, it is a drain on the mapper, since it is consuming memory and performing unnecessary hash lookups -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-291) [Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful
[ https://issues.apache.org/jira/browse/HIVE-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-291: Status: Patch Available (was: Open) have not run the patch on the main cluster yet - dont want to bring the cluster down in case of a bug on a long weekend. will merge only after testing with large data [Hive] map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful -- Key: HIVE-291 URL: https://issues.apache.org/jira/browse/HIVE-291 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: 291.1.txt Map-side aggregation should be automatically disabled at run-time if it is not turning out to be useful. If map-side aggregation is not reducing the number of output rows, it is a drain on the mapper, since it is consuming memory and performing unnecessary hash lookups -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
JIRA_291.1.txt_UNIT_TEST_SUCCEEDED
SUCCESS: BUILD AND UNIT TEST using PATCH 291.1.txt PASSED!!
[jira] Commented: (HIVE-270) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types
[ https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12673776#action_12673776 ] Joydeep Sen Sarma commented on HIVE-270: a couple of things i missed in first review: - where we get numberformatexception in lazy number parsing - should we return serdeexception instead? I am just worried that by sending them across as nulls - we are totally hiding any errors from the user. right now the serdeexceptions just go into a hadoop counter - but perhaps later we can surface them to the user (and error out job on threshold number of exceptions) - can we replace all occurences of metadatatypedserde? it seems that there are quite a few places in the code that are still using it (see planutils.getdefaulttabledesc for instance - that goes into a bunch of places - like script operator, fetch task, filesink) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types Key: HIVE-270 URL: https://issues.apache.org/jira/browse/HIVE-270 Project: Hadoop Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Zheng Shao Assignee: Zheng Shao Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch, HIVE-270.5.patch We want to add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types. This SerDe will share the same format as MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol to be backward compatible. This SerDe will be used to replace the default table SerDe, and the SerDe used to communicate with user scripts. For simplicity, we don't plan to support nested structure with this SerDe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-290) Add back generator for complex.seq
[ https://issues.apache.org/jira/browse/HIVE-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12673778#action_12673778 ] Zheng Shao commented on HIVE-290: - These 2 files are a single patch. svn diff does not include changes to binary files so I have to upload it separately. Add back generator for complex.seq -- Key: HIVE-290 URL: https://issues.apache.org/jira/browse/HIVE-290 Project: Hadoop Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.2.0, 0.3.0 Reporter: Zheng Shao Attachments: complex.seq, HIVE-290.1.patch A generator for data/files/complex.seq was removed because it had dependencies on org.apache.hive.serde package. We need to add it back so it's easier to generate new versions of data/files/complex.seq if necessary. In order to generate data/file/complex.seq, we can run ant gen-testdata. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-270) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types
[ https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12673797#action_12673797 ] Zheng Shao commented on HIVE-270: - This transaction is aimed for better performance with the same semantics. For 1, DynamicSerDe with TCTLSeparatedProtocol has been doing the same thing for a long time. Let's open another jira if this is needed? For 2. I've opened a new jira HIVE-292 for it. Let's do it step-by-step to reduce risks. Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types Key: HIVE-270 URL: https://issues.apache.org/jira/browse/HIVE-270 Project: Hadoop Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Zheng Shao Assignee: Zheng Shao Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch, HIVE-270.5.patch We want to add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types. This SerDe will share the same format as MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol to be backward compatible. This SerDe will be used to replace the default table SerDe, and the SerDe used to communicate with user scripts. For simplicity, we don't plan to support nested structure with this SerDe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-270) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types
[ https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12673808#action_12673808 ] Joydeep Sen Sarma commented on HIVE-270: sure - ok to address later. +1 Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types Key: HIVE-270 URL: https://issues.apache.org/jira/browse/HIVE-270 Project: Hadoop Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Zheng Shao Assignee: Zheng Shao Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch, HIVE-270.5.patch We want to add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types. This SerDe will share the same format as MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol to be backward compatible. This SerDe will be used to replace the default table SerDe, and the SerDe used to communicate with user scripts. For simplicity, we don't plan to support nested structure with this SerDe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-293) report deserialize exceptions from serde's via exceptions
report deserialize exceptions from serde's via exceptions - Key: HIVE-293 URL: https://issues.apache.org/jira/browse/HIVE-293 Project: Hadoop Hive Issue Type: Bug Reporter: Joydeep Sen Sarma lazyserde and dynamicserde should report exceptions on number (and other) parsing errors so higher layers can take the correct action -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
