[
https://issues.apache.org/jira/browse/YARN-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509862#comment-14509862
]
Sangjin Lee commented on YARN-3437:
-----------------------------------
Thanks for your comments [~zjshen].
{quote}
1. The YARN code is nearly duplicate with YARN-3390. As YARN-3390 is almost
ready, we can get that patch in first.
2. Move this jira to MR project, only retain the MR code in the patch and do
some minor rebase according to YARN-3390.
{quote}
Let's do this. While I was working on YARN-3438, I'm realizing that for the
performance tests it is probably OK to use the TimelineCollectors directly and
bypass the TimelineCollectorManager altogether. If we do that, then this could
become purely a MR patch. I'll update this patch to remove the use of
TimelineCollectorManager and move this JIRA to MAPREDUCE. How's that sound?
{quote}
3. TimelineServicePerformanceTest is in different package and has the different
name. Hopefully it won't conflict with YARN-2556. So once YARN-2556 gets
committed, we just need to refactor TimelineServicePerformanceTest to reuse
YARN-2556 code. BTW, can we put TimelineServicePerformanceTest into the same
package of TimelineServicePerformance in YARN-2556, and rename it to
TimelineServicePerformanceTestv2?
{quote}
That's fine. I'll move it back to the same package.
{quote}
W.R.T to the patch, I'm a bit concerned that the write which contains one event
per entity is not so typical to represent real use case. And configuration and
metrics are even not covered. Is it more realistic to write an entity with 10
events and 10 metrics, which have 100 points in the time series?
And one nit in the patch: entity.setEntityType("TEZ_DAG_ID");. How about not
mentioning TEZ in the MR code?
{quote}
Note that this is adding simple entity writes. The more realistic part of the
test is coming in YARN-3438 (I'm nearly finished with that), and it will have
multiple levels of entities as well as metrics and configuration. We would use
that one for more realistic load whereas we could keep this mode as a simpler
test. Thoughts?
I'll change the name of the entity to be something else.
> convert load test driver to timeline service v.2
> ------------------------------------------------
>
> Key: YARN-3437
> URL: https://issues.apache.org/jira/browse/YARN-3437
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: YARN-3437.001.patch, YARN-3437.002.patch,
> YARN-3437.003.patch, YARN-3437.004.patch
>
>
> This subtask covers the work for converting the proposed patch for the load
> test driver (YARN-2556) to work with the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
