[
https://issues.apache.org/jira/browse/YARN-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544334#comment-14544334
]
Sangjin Lee commented on YARN-2556:
-----------------------------------
Thanks for the latest patch [~lichangleo]!
You might want to take a look at MAPREDUCE-6335 and MAPREDUCE-6337 to see some
of the changes and comments made in there. We'll need to consider how we can
reconcile these two. It would be *GREAT* if we have this patch in the form so
that the future reconciliation would be very straightforward. The concerns
might involve having a place to switch between the timeline service v1 and
timeline service v.2, and adopting refactoring that was done in the v.2 test.
For example, you'll see that in MAPREDUCE-6337 we separated the actual entity
creation logic almost out of the test driver itself. It'd be ideal if the test
driver is agnostic of the timeline service v.1 or v.2, and those are deferred
to the actual mappers. Again, not everything can be cleanly separated, but to
the extent we can, it would be great. Thanks!
By the way, this might need to be moved to the MAPREDUCE project as this
touches the mapreduce code although it is supposed to test the timeline service.
Some more specific comments:
(TimelineServerPerformance.java)
- l.45: unused incorrect import
- l.54-57: constants should follow the naming convention (upper case)
- l.64: it was pointed out on the v.2 test, but using "Tez" here is rather
awkward as this is in the mapreduce code; can we use a generic name?
- l.89: nit: indentation
- l.245: why is System.runFinalization() necessary? I don't see a compelling
reason why it should be called here (it was also pointed out in the other JIRA)
- l.254: bug: i++ => t++
(MapredTestDriver.java)
- it looks like there are a lot of whitespace changes here; let's not introduce
more changes...
> Tool to measure the performance of the timeline server
> ------------------------------------------------------
>
> Key: YARN-2556
> URL: https://issues.apache.org/jira/browse/YARN-2556
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Jonathan Eagles
> Assignee: Chang Li
> Labels: BB2015-05-TBR
> Attachments: YARN-2556-WIP.patch, YARN-2556-WIP.patch,
> YARN-2556.1.patch, YARN-2556.2.patch, YARN-2556.3.patch, YARN-2556.4.patch,
> YARN-2556.5.patch, YARN-2556.6.patch, YARN-2556.7.patch, YARN-2556.8.patch,
> YARN-2556.patch, yarn2556.patch, yarn2556.patch, yarn2556_wip.patch
>
>
> We need to be able to understand the capacity model for the timeline server
> to give users the tools they need to deploy a timeline server with the
> correct capacity.
> I propose we create a mapreduce job that can measure timeline server write
> and read performance. Transactions per second, I/O for both read and write
> would be a good start.
> This could be done as an example or test job that could be tied into gridmix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
