[
https://issues.apache.org/jira/browse/DRILL-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494075#comment-16494075
]
Pritesh Maker commented on DRILL-6449:
--------------------------------------
[~jaydlowrider] could you provide some steps to reproduce this issue (maybe
attach some sample data files and logs)?
> Coordination issue between drillbits
> ------------------------------------
>
> Key: DRILL-6449
> URL: https://issues.apache.org/jira/browse/DRILL-6449
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.13.0
> Environment: My POC info:
> * 2 EC2 "r4.xlarge" with 30G of memory
> * both are running zookeeper for load distribution
> * both are running drillbits.
> * Storage
> ** S3, dfs
> * Connection drivers: JDBC
> * File Types/info: Json, max size between: 7MB - 75MB, and around 2200
> different filles
>
> Reporter: Melvin Ramos
> Priority: Major
>
> hello
> We are evaluating apache drill 1.13.0 to be used on our data lake initiative.
>
> Problem:
> I was able to select data when I ran small (around 3-50 files) query data
> which is fine, but when I collapse all on month (2200 different json files) I
> get into problems.
> on initial run it looks like its doing its work, but after executing the
> actual work, both drillbits seems to wait on each other and see when they are
> done, and nothing returns on my SQL analyzer query (Dbeaver)
>
> Here are last few lines after the run.
> Drill Bit 1
>
> {code:java}
> // code placeholder
> long_value: 380172
> }
> wait_nanos: 42213101
> }
> start_time: 1527341819358
> end_time: 1527341823288
> memory_used: 0
> max_memory_used: 5997952
> endpoint {
> address: "ip-10-0-2-209.us-west-2.compute.internal"
> user_port: 31010
> control_port: 31011
> data_port: 31012
> version: "1.13.0"
> state: ONLINE
> }
> }
> handle {
> query_id {
> part1: 2663488914051308194
> part2: -2037774217156077667
> }
> major_fragment_id: 1
> minor_fragment_id: 4
> }
> 2018-05-26 13:37:03,290 [BitServer-9] DEBUG
> o.a.d.exec.work.foreman.QueryManager - Foreman is still waiting for
> completion message from 1 nodes containing 4 fragments
> {code}
>
> Drill Bit 2
>
> {code:java}
> // code placeholder
> al.config.Limit
> 2018-05-26 13:37:03,197 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:7] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.physical.config.SelectionVectorRemover
> 2018-05-26 13:37:03,197 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:7] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.physical.config.SingleSender
> 2018-05-26 13:37:03,197 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:7] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 24f69d0e-3938-1aa2-e3b8-5f2565f6579d:1:7: State change requested RUNNING -->
> FINISHED
> 2018-05-26 13:37:03,198 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:7] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 24f69d0e-3938-1aa2-e3b8-5f2565f6579d:1:7: State to report: FINISHED
> 2018-05-26 13:37:03,198 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:7] DEBUG
> o.a.d.e.w.f.FragmentStatusReporter - Closing
> org.apache.drill.exec.work.fragment.FragmentStatusReporter@2552ae4c
> 2018-05-26 13:37:03,198 [drill-executor-3] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-05-26 13:37:03,199 [drill-executor-3] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 1 running fragments to
> complete before shutting down
> 2018-05-26 13:37:03,199 [drill-executor-3] INFO
> o.apache.drill.exec.work.WorkManager - New Fragments or queries are added
> while drillbit is Shutting down
> 2018-05-26 13:37:03,287 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.e.physical.impl.BaseRootExec - closed operator 793318686
> 2018-05-26 13:37:03,287 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.store.dfs.easy.EasySubScan
> 2018-05-26 13:37:03,287 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.physical.config.Limit
> 2018-05-26 13:37:03,287 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.physical.config.SelectionVectorRemover
> 2018-05-26 13:37:03,287 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.exec.ops.OperatorContextImpl - Closing context for
> org.apache.drill.exec.physical.config.SingleSender
> 2018-05-26 13:37:03,288 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 24f69d0e-3938-1aa2-e3b8-5f2565f6579d:1:4: State change requested RUNNING -->
> FINISHED
> 2018-05-26 13:37:03,288 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 24f69d0e-3938-1aa2-e3b8-5f2565f6579d:1:4: State to report: FINISHED
> 2018-05-26 13:37:03,288 [24f69d0e-3938-1aa2-e3b8-5f2565f6579d:frag:1:4] DEBUG
> o.a.d.e.w.f.FragmentStatusReporter - Closing
> org.apache.drill.exec.work.fragment.FragmentStatusReporter@166e8e76
> {code}
>
> JVM Environment Settings:
> {code:java}
> // code placeholder
> export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS -Dbounds=false
> -Ddrill.exec.memory.enable_unsafe_bounds_check=true"
> # Amount of heap memory for the Drillbit process. Values are those supported
> by
> # the Java -Xms option. The default is 4G.
> export DRILL_HEAP=${DRILL_HEAP:-"20G"}
> # Maximum amount of direct memory to allocate to the Drillbit in the format
> # supported by -XX:MaxDirectMemorySize. Default is 8G.
> export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"16G"}
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
