[jira] [Comment Edited] (MAPREDUCE-6486) File{Input|Output}Format counters not updated when using viewfs
[
https://issues.apache.org/jira/browse/MAPREDUCE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741494#comment-15741494
]
Vinayakumar B edited comment on MAPREDUCE-6486 at 12/12/16 9:49 AM:
Since ViewFS is just a wrapper over real filesystems, it would be better just
ignore these stats and dont show it in final counters list.
{{FileSystem.getAllStatistics()}} can filter stats for all filesystems with
child filesystems, or just viewfs.
What do you think [~cnauroth]?
was (Author: vinayrpet):
Since ViewFS is just a wrapper over real filesystems, it would be better just
ignore these stats and dont show it in final counters list.
Since these counters are already updated for resolved filesystems.
{{FileSystem.getAllStatistics()}} can filter stats for all filesystems with
child filesystems, or just viewfs.
What do you think [~cnauroth]?
> File{Input|Output}Format counters not updated when using viewfs
> ---
>
> Key: MAPREDUCE-6486
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6486
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, task
>Reporter: Laurent Goujon
>
> MapReduce tasks rely on filesystem stats to update FileInputFormatCounter and
> FileOutputFormatCounter bytes counters.
> But if the filesystem is viewfs, stats are not updated and counters are zero.
> However the resolved filesystem for the input and output paths have their
> stats updated, and should be used by the framework (which could be done by
> resolving the path first).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6486) File{Input|Output}Format counters not updated when using viewfs
[
https://issues.apache.org/jira/browse/MAPREDUCE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741494#comment-15741494
]
Vinayakumar B commented on MAPREDUCE-6486:
--
Since ViewFS is just a wrapper over real filesystems, it would be better just
ignore these stats and dont show it in final counters list.
Since these counters are already updated for resolved filesystems.
{{FileSystem.getAllStatistics()}} can filter stats for all filesystems with
child filesystems, or just viewfs.
What do you think [~cnauroth]?
> File{Input|Output}Format counters not updated when using viewfs
> ---
>
> Key: MAPREDUCE-6486
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6486
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, task
>Reporter: Laurent Goujon
>
> MapReduce tasks rely on filesystem stats to update FileInputFormatCounter and
> FileOutputFormatCounter bytes counters.
> But if the filesystem is viewfs, stats are not updated and counters are zero.
> However the resolved filesystem for the input and output paths have their
> stats updated, and should be used by the framework (which could be done by
> resolving the path first).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6666) Support MultiThreads in a Map and Distribution of files in NNBench
[ https://issues.apache.org/jira/browse/MAPREDUCE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-: - Summary: Support MultiThreads in a Map and Distribution of files in NNBench (was: Support Distribution of files to multiple directories generated by NNBench) > Support MultiThreads in a Map and Distribution of files in NNBench > -- > > Key: MAPREDUCE- > URL: https://issues.apache.org/jira/browse/MAPREDUCE- > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > > Support Distribution of files to multiple directories generated by NNBench. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215357#comment-15215357 ] Vinayakumar B commented on MAPREDUCE-6662: -- Thanks [~ajisakaa] and [~rohithsharma]. > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Fix For: 2.9.0 > > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch, MAPREDUCE-6662.04.patch, MAPREDUCE-6662.05.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213721#comment-15213721 ] Vinayakumar B commented on MAPREDUCE-6662: -- Hi [~ajisakaa], Do you mind taking a look at this? > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch, MAPREDUCE-6662.04.patch, MAPREDUCE-6662.05.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212972#comment-15212972 ] Vinayakumar B commented on MAPREDUCE-6662: -- One ASF License Warning is related to HDFS-9005. Test failures are not related. > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch, MAPREDUCE-6662.04.patch, MAPREDUCE-6662.05.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Attachment: MAPREDUCE-6662.05.patch Updated again > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch, MAPREDUCE-6662.04.patch, MAPREDUCE-6662.05.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Attachment: MAPREDUCE-6662.04.patch Updated > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch, MAPREDUCE-6662.04.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Attachment: MAPREDUCE-6662.03.patch Updated with some more. > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch, > MAPREDUCE-6662.03.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Attachment: MAPREDUCE-6662.02.patch Found some more. > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch, MAPREDUCE-6662.02.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211300#comment-15211300 ] Vinayakumar B commented on MAPREDUCE-6662: -- Uploaded a simple patch to generate the test files in 'already excluded' "target" directory. > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Status: Patch Available (was: Open) > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6662) Clear ASF Warnings on test data files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-6662: - Attachment: MAPREDUCE-6662.01.patch Uploaded a simple patch to generate the test files in > Clear ASF Warnings on test data files > - > > Key: MAPREDUCE-6662 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Minor > Attachments: MAPREDUCE-6662.01.patch > > > In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as > '-1 Overall'. > These warnings are from the files generated in tests. > It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6662) Clear ASF Warnings on test data files
Vinayakumar B created MAPREDUCE-6662: Summary: Clear ASF Warnings on test data files Key: MAPREDUCE-6662 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6662 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Vinayakumar B Assignee: Vinayakumar B Priority: Minor In all MAPREDUCE QA runs, '14 ASF Warnings' will mark the entire QA result as '-1 Overall'. These warnings are from the files generated in tests. It will be good to see green +1s in QA report than RED -1(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5948) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well
[ https://issues.apache.org/jira/browse/MAPREDUCE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570486#comment-14570486 ] Vinayakumar B commented on MAPREDUCE-5948: -- Sure, Go ahead :) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well -- Key: MAPREDUCE-5948 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5948 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2, 0.23.9, 2.2.0 Environment: CDH3U2 Redhat linux 5.7 Reporter: Kris Geusebroek Assignee: Rushabh S Shah Priority: Critical Attachments: HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch Having defined a recorddelimiter of multiple bytes in a new InputFileFormat sometimes has the effect of skipping records from the input. This happens when the input splits are split off just after a recordseparator. Starting point for the next split would be non zero and skipFirstLine would be true. A seek into the file is done to start - 1 and the text until the first recorddelimiter is ignored (due to the presumption that this record is already handled by the previous maptask). Since the re ord delimiter is multibyte the seek only got the last byte of the delimiter into scope and its not recognized as a full delimiter. So the text is skipped until the next delimiter (ignoring a full record!!) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539343#comment-14539343
]
Vinayakumar B commented on MAPREDUCE-6360:
--
+1 for the latest patch.
Waiting for jenkins
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch,
MAPREDUCE-6360-3.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated MAPREDUCE-6360:
-
Resolution: Pending Closed
Fix Version/s: 2.8.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Fix For: 2.8.0
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch,
MAPREDUCE-6360-3.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539427#comment-14539427
]
Vinayakumar B commented on MAPREDUCE-6360:
--
Committed to trunk and branch-2.
Thanks [~arshad.mohammad] for the contribution.
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch,
MAPREDUCE-6360-3.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B reopened MAPREDUCE-6360:
--
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Fix For: 2.8.0
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch,
MAPREDUCE-6360-3.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B resolved MAPREDUCE-6360.
--
Resolution: Fixed
Resolving as fixed.
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Fix For: 2.8.0
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch,
MAPREDUCE-6360-3.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated MAPREDUCE-6360:
-
Description:
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But eclipse
catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
was:
MAPREDUCE-6192 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But eclipse
catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Vinayakumar B
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6360) TestHdfsConfigFields is placed in wrong dir, introducing compile error
Vinayakumar B created MAPREDUCE-6360:
Summary: TestHdfsConfigFields is placed in wrong dir, introducing
compile error
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
HDFS-7559 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But eclipse
catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6360) TestHdfsConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B reassigned MAPREDUCE-6360:
Assignee: Vinayakumar B
TestHdfsConfigFields is placed in wrong dir, introducing compile error
--
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Vinayakumar B
HDFS-7559 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated MAPREDUCE-6360:
-
Summary: TestMapreduceConfigFields is placed in wrong dir, introducing
compile error (was: TestHdfsConfigFields is placed in wrong dir, introducing
compile error)
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Vinayakumar B
HDFS-7559 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated MAPREDUCE-6360:
-
Description:
MAPREDUCE-6192 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But eclipse
catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
was:
HDFS-7559 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But eclipse
catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Vinayakumar B
MAPREDUCE-6192 has introduced a Test file {{TestHdfsConfigFields}}
which was committed in package {{org.apache.hadoop.tools}}
But the package declaration inside file is {{org.apache.hadoop.hdfs.tools}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestHdfsConfigFields}} to correct package
{{org.apache.hadoop.hdfs.tools}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539263#comment-14539263
]
Vinayakumar B commented on MAPREDUCE-6360:
--
bq. it might be better to move into the
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce
directory
Yes, thats sounds reasonable to be as well.
[~arshad.mohammad], can you post one more patch which moves the file itself
instead of importing other things to current file. ?
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Attachments: MAPREDUCE-6360-1.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6360) TestMapreduceConfigFields is placed in wrong dir, introducing compile error
[
https://issues.apache.org/jira/browse/MAPREDUCE-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539302#comment-14539302
]
Vinayakumar B commented on MAPREDUCE-6360:
--
Patch didnt include the deleted file. This will be considered as new file. not
moved file.
Plz check again.
TestMapreduceConfigFields is placed in wrong dir, introducing compile error
---
Key: MAPREDUCE-6360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6360
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Vinayakumar B
Assignee: Arshad Mohammad
Attachments: MAPREDUCE-6360-1.patch, MAPREDUCE-6360-2.patch
MAPREDUCE-6192 has introduced a Test file {{TestMapreduceConfigFields }}
which was committed in package {{org.apache.hadoop.mapred}}
But the package declaration inside file is {{org.apache.hadoop.mapreduce}}.
By surprise, this is not giving any compile errors in maven build. But
eclipse catches it.
So move {{TestMapreduceConfigFields }} to correct package
{{org.apache.hadoop.mapreduce}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5948) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well
[ https://issues.apache.org/jira/browse/MAPREDUCE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated MAPREDUCE-5948: - Assignee: Rushabh S Shah (was: Vinayakumar B) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well -- Key: MAPREDUCE-5948 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5948 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2, 0.23.9, 2.2.0 Environment: CDH3U2 Redhat linux 5.7 Reporter: Kris Geusebroek Assignee: Rushabh S Shah Priority: Critical Attachments: HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch Having defined a recorddelimiter of multiple bytes in a new InputFileFormat sometimes has the effect of skipping records from the input. This happens when the input splits are split off just after a recordseparator. Starting point for the next split would be non zero and skipFirstLine would be true. A seek into the file is done to start - 1 and the text until the first recorddelimiter is ignored (due to the presumption that this record is already handled by the previous maptask). Since the re ord delimiter is multibyte the seek only got the last byte of the delimiter into scope and its not recognized as a full delimiter. So the text is skipped until the next delimiter (ignoring a full record!!) -- This message was sent by Atlassian JIRA (v6.2#6252)
