[
https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551238#comment-14551238
]
Li Lu commented on YARN-3411:
-----------------------------
Hi [~vrushalic], thanks for working on this! I just got back and looked at the
latest (v6) patch. In general I think it's quite close. I just have a few quick
comments.
# I notice you made {{EntityColumnFamilyDetails}} and
{{EntityInfoColumnDetails}} to be public. Even though we're adding @Private
tags to them, we may still want to use the strictest access modifier possible.
If they're fine to be visible to package only, maybe we can use default, rather
than public for them? (I tried with my local version and seems like they worked
fine, but not sure if I missed anything. )
# I share the same concern as [~zjshen] on schema creation. Having a separate
step to set up the HBase data schema appears to add some workload to Hadoop
deployments. So, although it's totally fine to have this step for our
performance benchmark, it will be great to keep this in mind when we
consolidate the storage part.
# Maybe it's helpful to move the entity creation steps of
TestHBaseTimelineWriterImpl to TestTimelineWriterImpl? In this way we can
improve test coverage for both writer implementations after this work.
> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
> Key: YARN-3411
> URL: https://issues.apache.org/jira/browse/YARN-3411
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Vrushali C
> Priority: Critical
> Attachments: ATSv2BackendHBaseSchemaproposal.pdf,
> YARN-3411-YARN-2928.001.patch, YARN-3411-YARN-2928.002.patch,
> YARN-3411-YARN-2928.003.patch, YARN-3411-YARN-2928.004.patch,
> YARN-3411-YARN-2928.005.patch, YARN-3411-YARN-2928.006.patch,
> YARN-3411.poc.2.txt, YARN-3411.poc.3.txt, YARN-3411.poc.4.txt,
> YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.7.txt,
> YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native
> HBase schema for the write path. Such a schema does not exclude using
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in
> terms of performance, scalability, usability, etc. and make a call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
