I have a table with multiple column family with possible same column names.
I want to use phoenix-spark plug in to select some of the fields, but it
returns a AnalysisException (details in the attached file)
public void testSpark(JavaSparkContext sc, String tableStr, String dataSrcUrl) {
//SparkContextBuilder.buildSparkContext("Simple Application", "local");
// One JVM can only have one Spark Context now
Map<String, String> options = new HashMap<String, String>();
SQLContext sqlContext = new SQLContext(sc);
options.put("zkUrl", dataSrcUrl);
options.put("table", tableStr);
log.info("Phoenix DB URL: " + dataSrcUrl + " tableStr: " + tableStr);
DataFrame df = null;
try {
df =
sqlContext.read().format("org.apache.phoenix.spark").options(options).load();
df.explain(true);
df.show();
df = df.select("I.CI", "I.FA");
//df = df.select("\"I\".\"CI\"", "\"I\".\"FA\""); // This gives the
same exception too
} catch (Exception ex) {
log.error("sql error: ", ex);
}
try {
log.info("Count By phoenix spark plugin: " + df.count());
} catch (Exception ex) {
log.error("dataframe error: ", ex);
}
}
I can see in the log that there is something like
10728 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat - Select
Statement: SELECT
"RID","I"."CI","I"."FA","I"."FPR","I"."FPT","I"."FR","I"."LAT","I"."LNG","I"."NCG","I"."NGPD","I"."VE","I"."VMJ","I"."VMR","I"."VP","I"."CSRE","I"."VIB","I"."IIICS","I"."LICSCD","I"."LEDC","I"."ARM","I"."FBM","I"."FTB","I"."NA2FR","I"."NA2PT","S"."AHDM","S"."ARTJ","S"."ATBM","S"."ATBMR","S"."ATBR","S"."ATBRR","S"."CS","S"."LAMT","S"."LTFCT","S"."LBMT","S"."LDTI","S"."LMT","S"."LMTN","S"."LMTR","S"."LPET","S"."LPORET","S"."LRMT","S"."LRMTP","S"."LRMTR","S"."LSRT","S"."LSST","S"."MHDMS0","S"."MHDMS1","S"."RFD","S"."RRN","S"."RRR","S"."TD","S"."TSM","S"."TC","S"."TPM","S"."LRMCT","S"."SS13FSK34","S"."LERMT","S"."LEMDMT","S"."AGTBRE","S"."SRM","S"."LTET","S"."TPMS","S"."TPMSM","S"."TM","S"."TMF","S"."TMFM","S"."NA2TLS","S"."NA2IT","S"."CWR","S"."BPR","S"."LR","S"."HLB","S"."NA2UFTBFR","S"."DT","S"."NA28ARE","S"."RM","S"."LMTB","S"."LRMTB","S"."RRB","P"."BADUC","P"."UAN","P"."BAPS","P"."BAS","P"."UAS","P"."BATBBR","P"."BBRI","P"."BLBR","P"."ULHT","P"."BLPST","P"."BLPT","P"."UTI","P"."UUC"
FROM TESTING.ENDPOINTS
But obviously, the column family is left out of the Dataframe column name
somewhere in the process.
Any fix for the problem?
Thanks
Xindian
9898 [INFO] main org.apache.spark.SparkContext - Starting job: count at
TestQuery.java:95
9901 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Registering RDD 5 (count at TestQuery.java:95)
9901 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Got job 1 (count at TestQuery.java:95) with 1 output partitions
9901 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Final stage: ResultStage 2 (count at TestQuery.java:95)
9901 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Parents of final stage: List(ShuffleMapStage 1)
9902 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Missing parents: List(ShuffleMapStage 1)
9903 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting ShuffleMapStage 1 (MapPartitionsRDD[5] at count at
TestQuery.java:95), which has no missing parents
9910 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_1 stored as values in memory (estimated size 10.9 KB, free 18.2
KB)
9911 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_1_piece0 stored as bytes in memory (estimated size 5.2 KB, free
23.3 KB)
9912 [INFO] dispatcher-event-loop-0 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_1_piece0 in memory on localhost:56810 (size: 5.2 KB, free:
5.1 GB)
9912 [INFO] dag-scheduler-event-loop org.apache.spark.SparkContext - Created
broadcast 1 from broadcast at DAGScheduler.scala:1006
9913 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting 1 missing tasks from ShuffleMapStage 1 (MapPartitionsRDD[5] at
count at TestQuery.java:95)
9914 [INFO] dag-scheduler-event-loop
org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 1.0 with 1 tasks
9916 [INFO] dispatcher-event-loop-1 org.apache.spark.scheduler.TaskSetManager
- Starting task 0.0 in stage 1.0 (TID 1, localhost, partition 0,PROCESS_LOCAL,
1911 bytes)
9916 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Running task 0.0 in stage 1.0 (TID 1)
10094 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateMutableProjection -
Code generated in 117.537099 ms
10109 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection -
Code generated in 7.281172 ms
10140 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateMutableProjection -
Code generated in 19.739571 ms
10150 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeRowJoiner -
Code generated in 4.854115 ms
10150 [INFO] dispatcher-event-loop-5 org.apache.spark.storage.BlockManagerInfo
- Removed broadcast_0_piece0 on localhost:56810 in memory (size: 2.4 KB, free:
5.1 GB)
10152 [INFO] Spark Context Cleaner org.apache.spark.ContextCleaner - Cleaned
accumulator 1
10158 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection -
Code generated in 6.082508 ms
10293 [INFO] Executor task launch worker-0
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD - closed connection
10327 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Finished task 0.0 in stage 1.0 (TID 1). 1405 bytes result sent to driver
10330 [INFO] task-result-getter-1 org.apache.spark.scheduler.TaskSetManager -
Finished task 0.0 in stage 1.0 (TID 1) in 415 ms on localhost (1/1)
10330 [INFO] task-result-getter-1 org.apache.spark.scheduler.TaskSchedulerImpl
- Removed TaskSet 1.0, whose tasks have all completed, from pool
10331 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- ShuffleMapStage 1 (count at TestQuery.java:95) finished in 0.417 s
10331 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- looking for newly runnable stages
10331 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- running: Set()
10332 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- waiting: Set(ResultStage 2)
10332 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- failed: Set()
10333 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting ResultStage 2 (MapPartitionsRDD[8] at count at TestQuery.java:95),
which has no missing parents
10336 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_2 stored as values in memory (estimated size 12.1 KB, free 28.2
KB)
10337 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_2_piece0 stored as bytes in memory (estimated size 5.6 KB, free
33.9 KB)
10338 [INFO] dispatcher-event-loop-7 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_2_piece0 in memory on localhost:56810 (size: 5.6 KB, free:
5.1 GB)
10338 [INFO] dag-scheduler-event-loop org.apache.spark.SparkContext - Created
broadcast 2 from broadcast at DAGScheduler.scala:1006
10338 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[8] at count
at TestQuery.java:95)
10338 [INFO] dag-scheduler-event-loop
org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 2.0 with 1 tasks
10342 [INFO] dispatcher-event-loop-0 org.apache.spark.scheduler.TaskSetManager
- Starting task 0.0 in stage 2.0 (TID 2, localhost, partition 0,NODE_LOCAL,
1999 bytes)
10342 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Running task 0.0 in stage 2.0 (TID 2)
10353 [INFO] Executor task launch worker-0
org.apache.spark.storage.ShuffleBlockFetcherIterator - Getting 1 non-empty
blocks out of 1 blocks
10353 [INFO] Executor task launch worker-0
org.apache.spark.storage.ShuffleBlockFetcherIterator - Started 0 remote
fetches in 2 ms
10375 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateMutableProjection -
Code generated in 5.233497 ms
10410 [INFO] Executor task launch worker-0
org.apache.spark.sql.catalyst.expressions.codegen.GenerateMutableProjection -
Code generated in 4.493135 ms
10418 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Finished task 0.0 in stage 2.0 (TID 2). 1830 bytes result sent to driver
10419 [INFO] task-result-getter-2 org.apache.spark.scheduler.TaskSetManager -
Finished task 0.0 in stage 2.0 (TID 2) in 78 ms on localhost (1/1)
10419 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- ResultStage 2 (count at TestQuery.java:95) finished in 0.079 s
10420 [INFO] task-result-getter-2 org.apache.spark.scheduler.TaskSchedulerImpl
- Removed TaskSet 2.0, whose tasks have all completed, from pool
10421 [INFO] main org.apache.spark.scheduler.DAGScheduler - Job 1 finished:
count at TestQuery.java:95, took 0.522130 s
10422 [INFO] main com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery - Count
By Jdbc: 2834
10425 [INFO] main com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery -
Phoenix DB URL:
jdbc:phoenix:luna-sdp-nms-01.davis.sensus.lab:2181:/hbase-unsecure tableStr:
TESTING.ENDPOINTS
10483 [INFO] main org.apache.spark.storage.MemoryStore - Block broadcast_3
stored as values in memory (estimated size 379.2 KB, free 413.1 KB)
10505 [INFO] main org.apache.spark.storage.MemoryStore - Block
broadcast_3_piece0 stored as bytes in memory (estimated size 27.3 KB, free
440.3 KB)
10505 [INFO] dispatcher-event-loop-3 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_3_piece0 in memory on localhost:56810 (size: 27.3 KB, free:
5.1 GB)
10506 [INFO] main org.apache.spark.SparkContext - Created broadcast 3 from
newAPIHadoopRDD at PhoenixRDD.scala:46
10591 [INFO] main org.apache.spark.storage.MemoryStore - Block broadcast_4
stored as values in memory (estimated size 407.1 KB, free 847.5 KB)
10607 [INFO] main org.apache.spark.storage.MemoryStore - Block
broadcast_4_piece0 stored as bytes in memory (estimated size 28.5 KB, free
876.0 KB)
10608 [INFO] dispatcher-event-loop-4 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_4_piece0 in memory on localhost:56810 (size: 28.5 KB, free:
5.1 GB)
10609 [INFO] main org.apache.spark.SparkContext - Created broadcast 4 from
newAPIHadoopRDD at PhoenixRDD.scala:46
10615 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat -
UseSelectColumns=true, selectColumnList.size()=92,
selectColumnList=RID,CI,FA,FPR,FPT,FR,LAT,LNG,NCG,NGPD,VE,VMJ,VMR,VP,CSRE,VIB,IIICS,LICSCD,LEDC,ARM,FBM,FTB,NA2FR,NA2PT,AHDM,ARTJ,ATBM,ATBMR,ATBR,ATBRR,CS,LAMT,LTFCT,LBMT,LDTI,LMT,LMTN,LMTR,LPET,LPORET,LRMT,LRMTP,LRMTR,LSRT,LSST,MHDMS0,MHDMS1,RFD,RRN,RRR,TD,TSM,TC,TPM,LRMCT,SS13FSK34,LERMT,LEMDMT,AGTBRE,SRM,LTET,TPMS,TPMSM,TM,TMF,TMFM,NA2TLS,NA2IT,CWR,BPR,LR,HLB,NA2UFTBFR,DT,NA28ARE,RM,LMTB,LRMTB,RRB,BADUC,UAN,BAPS,BAS,UAS,BATBBR,BBRI,BLBR,ULHT,BLPST,BLPT,UTI,UUC
== Parsed Logical Plan ==
Relation[RID#99,CI#100,FA#101,FPR#102,FPT#103,FR#104,LAT#105,LNG#106,NCG#107,NGPD#108,VE#109,VMJ#110,VMR#111,VP#112,CSRE#113,VIB#114,IIICS#115,LICSCD#116,LEDC#117,ARM#118,FBM#119,FTB#120,NA2FR#121,NA2PT#122,AHDM#123,ARTJ#124,ATBM#125,ATBMR#126,ATBR#127,ATBRR#128,CS#129,LAMT#130,LTFCT#131,LBMT#132,LDTI#133,LMT#134,LMTN#135,LMTR#136,LPET#137,LPORET#138,LRMT#139,LRMTP#140,LRMTR#141,LSRT#142,LSST#143,MHDMS0#144,MHDMS1#145,RFD#146,RRN#147,RRR#148,TD#149,TSM#150,TC#151,TPM#152,LRMCT#153,SS13FSK34#154,LERMT#155,LEMDMT#156,AGTBRE#157,SRM#158,LTET#159,TPMS#160,TPMSM#161,TM#162,TMF#163,TMFM#164,NA2TLS#165,NA2IT#166,CWR#167,BPR#168,LR#169,HLB#170,NA2UFTBFR#171,DT#172,NA28ARE#173,RM#174,LMTB#175,LRMTB#176,RRB#177,BADUC#178,UAN#179,BAPS#180,BAS#181,UAS#182,BATBBR#183,BBRI#184,BLBR#185,ULHT#186,BLPST#187,BLPT#188,UTI#189,UUC#190]
PhoenixRelation(TESTING.ENDPOINTS,jdbc:phoenix:luna-sdp-nms-01.davis.sensus.lab:2181:/hbase-unsecure)
== Analyzed Logical Plan ==
RID: int, CI: string, FA: int, FPR: int, FPT: int, FR: int, LAT: double, LNG:
double, NCG: smallint, NGPD: timestamp, VE: smallint, VMJ: smallint, VMR:
smallint, VP: smallint, CSRE: boolean, VIB: boolean, IIICS: boolean, LICSCD:
timestamp, LEDC: timestamp, ARM: int, FBM: int, FTB: int, NA2FR: int, NA2PT:
int, AHDM: float, ARTJ: float, ATBM: float, ATBMR: float, ATBR: float, ATBRR:
float, CS: int, LAMT: timestamp, LTFCT: timestamp, LBMT: timestamp, LDTI: int,
LMT: timestamp, LMTN: timestamp, LMTR: timestamp, LPET: timestamp, LPORET:
timestamp, LRMT: timestamp, LRMTP: timestamp, LRMTR: timestamp, LSRT:
timestamp, LSST: timestamp, MHDMS0: smallint, MHDMS1: smallint, RFD: boolean,
RRN: binary, RRR: binary, TD: boolean, TSM: int, TC: int, TPM: smallint, LRMCT:
timestamp, SS13FSK34: boolean, LERMT: timestamp, LEMDMT: timestamp, AGTBRE:
float, SRM: int, LTET: timestamp, TPMS: smallint, TPMSM: smallint, TM: int,
TMF: int, TMFM: int, NA2TLS: int, NA2IT: boolean, CWR: boolean, BPR: boolean,
LR: boolean, HLB: boolean, NA2UFTBFR: boolean, DT: int, NA28ARE: boolean, RM:
int, LMTB: timestamp, LRMTB: timestamp, RRB: binary, BADUC: binary, UAN:
binary, BAPS: binary, BAS: binary, UAS: binary, BATBBR: binary, BBRI: binary,
BLBR: binary, ULHT: binary, BLPST: binary, BLPT: binary, UTI: binary, UUC:
binary
Relation[RID#99,CI#100,FA#101,FPR#102,FPT#103,FR#104,LAT#105,LNG#106,NCG#107,NGPD#108,VE#109,VMJ#110,VMR#111,VP#112,CSRE#113,VIB#114,IIICS#115,LICSCD#116,LEDC#117,ARM#118,FBM#119,FTB#120,NA2FR#121,NA2PT#122,AHDM#123,ARTJ#124,ATBM#125,ATBMR#126,ATBR#127,ATBRR#128,CS#129,LAMT#130,LTFCT#131,LBMT#132,LDTI#133,LMT#134,LMTN#135,LMTR#136,LPET#137,LPORET#138,LRMT#139,LRMTP#140,LRMTR#141,LSRT#142,LSST#143,MHDMS0#144,MHDMS1#145,RFD#146,RRN#147,RRR#148,TD#149,TSM#150,TC#151,TPM#152,LRMCT#153,SS13FSK34#154,LERMT#155,LEMDMT#156,AGTBRE#157,SRM#158,LTET#159,TPMS#160,TPMSM#161,TM#162,TMF#163,TMFM#164,NA2TLS#165,NA2IT#166,CWR#167,BPR#168,LR#169,HLB#170,NA2UFTBFR#171,DT#172,NA28ARE#173,RM#174,LMTB#175,LRMTB#176,RRB#177,BADUC#178,UAN#179,BAPS#180,BAS#181,UAS#182,BATBBR#183,BBRI#184,BLBR#185,ULHT#186,BLPST#187,BLPT#188,UTI#189,UUC#190]
PhoenixRelation(TESTING.ENDPOINTS,jdbc:phoenix:luna-sdp-nms-01.davis.sensus.lab:2181:/hbase-unsecure)
== Optimized Logical Plan ==
Relation[RID#99,CI#100,FA#101,FPR#102,FPT#103,FR#104,LAT#105,LNG#106,NCG#107,NGPD#108,VE#109,VMJ#110,VMR#111,VP#112,CSRE#113,VIB#114,IIICS#115,LICSCD#116,LEDC#117,ARM#118,FBM#119,FTB#120,NA2FR#121,NA2PT#122,AHDM#123,ARTJ#124,ATBM#125,ATBMR#126,ATBR#127,ATBRR#128,CS#129,LAMT#130,LTFCT#131,LBMT#132,LDTI#133,LMT#134,LMTN#135,LMTR#136,LPET#137,LPORET#138,LRMT#139,LRMTP#140,LRMTR#141,LSRT#142,LSST#143,MHDMS0#144,MHDMS1#145,RFD#146,RRN#147,RRR#148,TD#149,TSM#150,TC#151,TPM#152,LRMCT#153,SS13FSK34#154,LERMT#155,LEMDMT#156,AGTBRE#157,SRM#158,LTET#159,TPMS#160,TPMSM#161,TM#162,TMF#163,TMFM#164,NA2TLS#165,NA2IT#166,CWR#167,BPR#168,LR#169,HLB#170,NA2UFTBFR#171,DT#172,NA28ARE#173,RM#174,LMTB#175,LRMTB#176,RRB#177,BADUC#178,UAN#179,BAPS#180,BAS#181,UAS#182,BATBBR#183,BBRI#184,BLBR#185,ULHT#186,BLPST#187,BLPT#188,UTI#189,UUC#190]
PhoenixRelation(TESTING.ENDPOINTS,jdbc:phoenix:luna-sdp-nms-01.davis.sensus.lab:2181:/hbase-unsecure)
== Physical Plan ==
Scan
PhoenixRelation(TESTING.ENDPOINTS,jdbc:phoenix:luna-sdp-nms-01.davis.sensus.lab:2181:/hbase-unsecure)[RID#99,CI#100,FA#101,FPR#102,FPT#103,FR#104,LAT#105,LNG#106,NCG#107,NGPD#108,VE#109,VMJ#110,VMR#111,VP#112,CSRE#113,VIB#114,IIICS#115,LICSCD#116,LEDC#117,ARM#118,FBM#119,FTB#120,NA2FR#121,NA2PT#122,AHDM#123,ARTJ#124,ATBM#125,ATBMR#126,ATBR#127,ATBRR#128,CS#129,LAMT#130,LTFCT#131,LBMT#132,LDTI#133,LMT#134,LMTN#135,LMTR#136,LPET#137,LPORET#138,LRMT#139,LRMTP#140,LRMTR#141,LSRT#142,LSST#143,MHDMS0#144,MHDMS1#145,RFD#146,RRN#147,RRR#148,TD#149,TSM#150,TC#151,TPM#152,LRMCT#153,SS13FSK34#154,LERMT#155,LEMDMT#156,AGTBRE#157,SRM#158,LTET#159,TPMS#160,TPMSM#161,TM#162,TMF#163,TMFM#164,NA2TLS#165,NA2IT#166,CWR#167,BPR#168,LR#169,HLB#170,NA2UFTBFR#171,DT#172,NA28ARE#173,RM#174,LMTB#175,LRMTB#176,RRB#177,BADUC#178,UAN#179,BAPS#180,BAS#181,UAS#182,BATBBR#183,BBRI#184,BLBR#185,ULHT#186,BLPST#187,BLPT#188,UTI#189,UUC#190]
10682 [INFO] main org.apache.spark.storage.MemoryStore - Block broadcast_5
stored as values in memory (estimated size 407.1 KB, free 1283.1 KB)
10695 [INFO] main org.apache.spark.storage.MemoryStore - Block
broadcast_5_piece0 stored as bytes in memory (estimated size 28.5 KB, free
1311.6 KB)
10695 [INFO] dispatcher-event-loop-5 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_5_piece0 in memory on localhost:56810 (size: 28.5 KB, free:
5.1 GB)
10696 [INFO] main org.apache.spark.SparkContext - Created broadcast 5 from
newAPIHadoopRDD at PhoenixRDD.scala:46
10700 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat -
UseSelectColumns=true, selectColumnList.size()=92,
selectColumnList=RID,CI,FA,FPR,FPT,FR,LAT,LNG,NCG,NGPD,VE,VMJ,VMR,VP,CSRE,VIB,IIICS,LICSCD,LEDC,ARM,FBM,FTB,NA2FR,NA2PT,AHDM,ARTJ,ATBM,ATBMR,ATBR,ATBRR,CS,LAMT,LTFCT,LBMT,LDTI,LMT,LMTN,LMTR,LPET,LPORET,LRMT,LRMTP,LRMTR,LSRT,LSST,MHDMS0,MHDMS1,RFD,RRN,RRR,TD,TSM,TC,TPM,LRMCT,SS13FSK34,LERMT,LEMDMT,AGTBRE,SRM,LTET,TPMS,TPMSM,TM,TMF,TMFM,NA2TLS,NA2IT,CWR,BPR,LR,HLB,NA2UFTBFR,DT,NA28ARE,RM,LMTB,LRMTB,RRB,BADUC,UAN,BAPS,BAS,UAS,BATBBR,BBRI,BLBR,ULHT,BLPST,BLPT,UTI,UUC
10727 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat -
UseSelectColumns=true, selectColumnList.size()=92,
selectColumnList=RID,CI,FA,FPR,FPT,FR,LAT,LNG,NCG,NGPD,VE,VMJ,VMR,VP,CSRE,VIB,IIICS,LICSCD,LEDC,ARM,FBM,FTB,NA2FR,NA2PT,AHDM,ARTJ,ATBM,ATBMR,ATBR,ATBRR,CS,LAMT,LTFCT,LBMT,LDTI,LMT,LMTN,LMTR,LPET,LPORET,LRMT,LRMTP,LRMTR,LSRT,LSST,MHDMS0,MHDMS1,RFD,RRN,RRR,TD,TSM,TC,TPM,LRMCT,SS13FSK34,LERMT,LEMDMT,AGTBRE,SRM,LTET,TPMS,TPMSM,TM,TMF,TMFM,NA2TLS,NA2IT,CWR,BPR,LR,HLB,NA2UFTBFR,DT,NA28ARE,RM,LMTB,LRMTB,RRB,BADUC,UAN,BAPS,BAS,UAS,BATBBR,BBRI,BLBR,ULHT,BLPST,BLPT,UTI,UUC
10728 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat - Select
Statement: SELECT
"RID","I"."CI","I"."FA","I"."FPR","I"."FPT","I"."FR","I"."LAT","I"."LNG","I"."NCG","I"."NGPD","I"."VE","I"."VMJ","I"."VMR","I"."VP","I"."CSRE","I"."VIB","I"."IIICS","I"."LICSCD","I"."LEDC","I"."ARM","I"."FBM","I"."FTB","I"."NA2FR","I"."NA2PT","S"."AHDM","S"."ARTJ","S"."ATBM","S"."ATBMR","S"."ATBR","S"."ATBRR","S"."CS","S"."LAMT","S"."LTFCT","S"."LBMT","S"."LDTI","S"."LMT","S"."LMTN","S"."LMTR","S"."LPET","S"."LPORET","S"."LRMT","S"."LRMTP","S"."LRMTR","S"."LSRT","S"."LSST","S"."MHDMS0","S"."MHDMS1","S"."RFD","S"."RRN","S"."RRR","S"."TD","S"."TSM","S"."TC","S"."TPM","S"."LRMCT","S"."SS13FSK34","S"."LERMT","S"."LEMDMT","S"."AGTBRE","S"."SRM","S"."LTET","S"."TPMS","S"."TPMSM","S"."TM","S"."TMF","S"."TMFM","S"."NA2TLS","S"."NA2IT","S"."CWR","S"."BPR","S"."LR","S"."HLB","S"."NA2UFTBFR","S"."DT","S"."NA28ARE","S"."RM","S"."LMTB","S"."LRMTB","S"."RRB","P"."BADUC","P"."UAN","P"."BAPS","P"."BAS","P"."UAS","P"."BATBBR","P"."BBRI","P"."BLBR","P"."ULHT","P"."BLPST","P"."BLPT","P"."UTI","P"."UUC"
FROM TESTING.ENDPOINTS
10744 [INFO] main org.apache.spark.SparkContext - Starting job: show at
TestQuery.java:49
10745 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Got job 2 (show at TestQuery.java:49) with 1 output partitions
10746 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Final stage: ResultStage 3 (show at TestQuery.java:49)
10746 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Parents of final stage: List()
10746 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Missing parents: List()
10746 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting ResultStage 3 (MapPartitionsRDD[25] at show at TestQuery.java:49),
which has no missing parents
10750 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_6 stored as values in memory (estimated size 13.1 KB, free
1324.7 KB)
10751 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_6_piece0 stored as bytes in memory (estimated size 5.5 KB, free
1330.3 KB)
10752 [INFO] dispatcher-event-loop-6 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_6_piece0 in memory on localhost:56810 (size: 5.5 KB, free:
5.1 GB)
10752 [INFO] dag-scheduler-event-loop org.apache.spark.SparkContext - Created
broadcast 6 from broadcast at DAGScheduler.scala:1006
10752 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting 1 missing tasks from ResultStage 3 (MapPartitionsRDD[25] at show
at TestQuery.java:49)
10752 [INFO] dag-scheduler-event-loop
org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 3.0 with 1 tasks
10758 [INFO] dispatcher-event-loop-7 org.apache.spark.scheduler.TaskSetManager
- Starting task 0.0 in stage 3.0 (TID 3, localhost, partition 0,PROCESS_LOCAL,
3875 bytes)
10758 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Running task 0.0 in stage 3.0 (TID 3)
10769 [INFO] Executor task launch worker-0 org.apache.spark.rdd.NewHadoopRDD
- Input split: org.apache.phoenix.mapreduce.PhoenixInputSplit@20b488
10844 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Finished task 0.0 in stage 3.0 (TID 3). 19120 bytes result sent to driver
10847 [INFO] task-result-getter-3 org.apache.spark.scheduler.TaskSetManager -
Finished task 0.0 in stage 3.0 (TID 3) in 95 ms on localhost (1/1)
10848 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- ResultStage 3 (show at TestQuery.java:49) finished in 0.095 s
10848 [INFO] task-result-getter-3 org.apache.spark.scheduler.TaskSchedulerImpl
- Removed TaskSet 3.0, whose tasks have all completed, from pool
10848 [INFO] main org.apache.spark.scheduler.DAGScheduler - Job 2 finished:
show at TestQuery.java:49, took 0.103505 s
+----+----+---------+---------+---------+---------+------------------+-------------------+----+--------------------+----+----+----+----+-----+-----+-----+--------------------+----+---+----+---------+-----+-----+---------+---------+----------+---------+---------+---------+---+--------------------+-----+--------------------+----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+-----+--------------------+----+--------------------+------+------+-----+--------------------+--------------------+-----+----+-----+----+-----+---------+--------------------+------+---------+----+----+----+-----+----+----+----+------+-----+-----+-----+-----+-----+---------+----+-------+----+----+-----+--------------------+----------------+--------------------+----------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+----------------+----------------+--------------------+--------------------+
| RID| CI| FA| FPR| FPT| FR| LAT|
LNG| NCG| NGPD| VE| VMJ| VMR| VP| CSRE| VIB|IIICS|
LICSCD|LEDC|ARM| FBM| FTB|NA2FR|NA2PT| AHDM| ARTJ|
ATBM| ATBMR| ATBR| ATBRR| CS| LAMT|LTFCT|
LBMT|LDTI| LMT| LMTN| LMTR|
LPET| LPORET| LRMT|LRMTP|
LRMTR|LSRT| LSST|MHDMS0|MHDMS1| RFD| RRN|
RRR| TD| TSM| TC| TPM|LRMCT|SS13FSK34|
LERMT|LEMDMT| AGTBRE| SRM|LTET|TPMS|TPMSM| TM| TMF|TMFM|NA2TLS|NA2IT| CWR|
BPR| LR| HLB|NA2UFTBFR| DT|NA28ARE| RM|LMTB|LRMTB| RRB|
BADUC| UAN| BAPS| BAS|
UAS| BATBBR| BBRI| BLBR|
ULHT| BLPST| BLPT| UTI|
UUC|
+----+----+---------+---------+---------+---------+------------------+-------------------+----+--------------------+----+----+----+----+-----+-----+-----+--------------------+----+---+----+---------+-----+-----+---------+---------+----------+---------+---------+---------+---+--------------------+-----+--------------------+----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+-----+--------------------+----+--------------------+------+------+-----+--------------------+--------------------+-----+----+-----+----+-----+---------+--------------------+------+---------+----+----+----+-----+----+----+----+------+-----+-----+-----+-----+-----+---------+----+-------+----+----+-----+--------------------+----------------+--------------------+----------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+----------------+----------------+--------------------+--------------------+
| 0|ACME|901112500|901143750|901131250|922000000| 35.86735568957524|
-78.86121107739518| 106|2016-10-14 00:03:...| 0| 3| 7|
0|false|false|false|2016-07-24 06:22:...|null| 0|null|922000000| null|
null|7739.4365|1486.2961| 11434.249|630341.56|16336.882|16336.882|
2|2016-11-08 12:23:...| null| null|null|2016-11-08
12:23:...|2016-11-08 12:23:...|2016-11-08 01:22:...|2016-11-08
11:33:...|2016-09-08 17:54:...|2016-07-27 03:23:...| null|2016-07-27
03:23:...|null|2016-07-24 06:22:...| 7740| 7740|false|[00 00 00 00 00
0...|[AA D5 AA 04 AD 5...|false| 1| 664| 240| null| false|
null| null| null| 60|null|null| null| 1|null|null| 2|
true|false|false|false|false| false| 47| false| 7|null| null|[00 00 00
00 00 0...| null|[00 00 00 13 01 4...| null|
null|[00 00 00 13 01 4...| null| null|
null|[00 00 00 1F 01 0...| null| null|[00 00 00 13
01 0...|[00 00 00 13 01 0...|
| 1|null| null| null| null| null| 1.0|
2.0|null|2016-09-13 14:03:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null|
null|122773.375| null| null| null| 2| null| null|
null|null|2016-11-04 16:19:...| null|
null| null| null| null| null|
null|null| null| -1| -1|false|
null| null|false|null| 36|null| null| false|
null| null| null|null|null|null| null|null|null|null| null| null| null|
null| null| null| null| 102| null|null|null| null| null|
null|[00 00 00 09 01 4...| null| null|[00 00
00 09 01 4...| null| null|
null|[00 00 00 0D 01 0...| null| null|[00 00 00 09 01
0...|[00 00 00 09 01 0...|
| 123|null| null| null| null| null| null|
null|null|2016-09-13 14:03:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null| null|
null| null| null| null| 4| null| null|
null|null|2016-09-13 10:08:...| null| null|
null| null| null| null|
null|null| null| -1| -1|false| null|
null|false|null| 4|null| null| null| null| null|
null|null|null|null| null|null|null|null| null| null| null| null| null|
null| null|null| null|null|null| null| null|
null|[00 00 00 09 01 4...| null| null|[00 00 00 09 01
4...| null| null| null|[00 00 00
0D 01 0...| null| null|[00 00 00 09 01 0...|[00 00 00 09
01 0...|
| 174|null| null| null| null| null| null|
null| 106|2016-11-06 00:03:...|null|null|null|null|false| null| null|
null|null| 0|null| null| null| null| null| null|
4805.9204| null| null| null| 1| null| null|
null|null|2016-11-05 08:41:...| null|
null| null| null| null| null|
null|null| null| -1| -1|false|
null| null|false|null| 3|null| null| null|
null| null| null|null|null|null| null|null|null|null| null| null| null|
null| null| null| null|null| null|null|null| null| null|
null|[00 00 00 09 01 4...| null| null|[00 00
00 09 01 4...| null| null|
null|[00 00 00 0D 01 0...| null| null|[00 00 00 09 01
0...|[00 00 00 09 01 0...|
| 304|null| null| null| null| null| null|
null|null|2016-09-10 00:03:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null| null|
null| null| null| null| 4| null| null|
null|null|2016-09-09 02:37:...| null| null|
null| null| null| null|
null|null| null| -1| -1|false| null|
null|false|null| 3|null| null| null| null| null|
null|null|null|null| null|null|null|null| null| null| null| null| null|
null| null|null| null|null|null| null| null|
null|[00 00 00 09 01 4...| null| null|[00 00 00 09 01
4...| null| null| null|[00 00 00
0D 01 0...| null| null|[00 00 00 09 01 0...|[00 00 00 09
01 0...|
| 333|null| null| null| null| null| null|
null|null|2016-07-22 00:03:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null| null|
132339.44|132344.53|500542.66| 427297.0| 4| null| null|
null|null|2016-08-03 11:00:...| null|2016-08-03
11:00:...| null| null|2016-08-02 02:26:...|
null|2016-08-02 02:26:...|null| null| -1| -1|false|
null| null|false|null| 63|null| null|
null|2016-08-02 02:26:...| null|500542.66|null|null|null| null|null|null|null|
null| null| null| null| null| null| null|null| null|null|null| null|
null| null|[00 00 00 0A 01 4...| null|
null|[00 00 00 0A 01 4...| null| null|
null|[00 00 00 0E 01 0...| null| null|[00 00
00 0E 01 0...|[00 00 00 0E 01 0...|
| 339|null| null| null| null| null| null|
null|null|2016-08-12 00:03:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null| null|
33091.37| 40441.54| null| null| 4| null| null|
null|null|2016-09-17 22:24:...| null|2016-09-17 22:24:...|
null| null|2016-09-16 03:19:...| null|2016-09-16
03:19:...|null| null| -1| -1|false| null|
null|false|null| 106|null| null| null|2016-09-16 03:19:...|
null| null|null|null|null| null|null|null|null| null| null| null| null|
null| null| null|null| null|null|null| null| null|
null|[00 00 00 09 01 4...| null| null|[00 00 00 09
01 4...| null| null| null|[00 00
00 0D 01 0...| null| null|[00 00 00 09 01 0...|[00 00 00
09 01 0...|
| 364|null| null| null| null| null| null|
null|null|2016-07-20 17:53:...|null|null|null|null| null| null| null|
null|null| 0|null| null| null| null| null| null|
47039.094|154673.62|154673.62|154673.62| 2| null| null|
null|null|2016-11-07 13:08:...| null|2016-11-06
08:06:...| null| null|2016-11-06 08:06:...|
null|2016-11-06 08:06:...|null| null| -1| -1|false|
null| null|false|null| 202|null| null|
null|2016-11-06 08:06:...| null|154673.62|null|null|null| null|null|null|null|
null| null| null| null| null| null| null|null| null|null|null| null|
null|[00 00 00 05 00]|[00 00 00 09 01 4...|[00 00 00 05 00]|[00 00
00 09 01 4...|[00 00 00 09 01 4...|[00 00 00 09 01 4...|[00 00 00 09 01
0...|[00 00 00 0D 01 0...|[00 00 00 0D 01 0...|[00 00 00 05 00]|[00 00 00 05
00]|[00 00 00 09 01 0...|[00 00 00 09 01 0...|
| 421|null|901212500|901243750|901231250|940212500|35.866798400878906|
-78.86139678955078|null|2016-08-11 00:03:...| 0| 5| 5|
5|false|false|false|2016-10-31 11:00:...|null| 0|null|940212500| null| null|
255.0| 0.0| 19666.121| null|23744.875| null| 2|
null| null| null|null|2016-11-08 12:41:...|2016-11-08 12:41:...|
null|2016-08-20 00:31:...| null|2016-11-08
12:41:...| null| null|null|2016-10-30 06:37:...| 255|
255|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false| 1| 502| 240|
null| null|2016-11-08 12:41:...| null|23744.875| 15|null|null| null|
9|null|null| null| null| null| null| null| null| null|null| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1001|null|901112500|901143750|901162500|940112500|
32.79841232299805|-116.78971862792969|null|2016-07-20 14:40:...| 0| 1| 3|
0| null|false|false|2016-11-04 10:15:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 879.8865|4737.9404| 879.8987|4737.9404|
2|2016-11-08 07:00:...| null| null|null|2016-11-08 12:30:...|
null|2016-11-08 12:01:...| null|
null|2016-11-08 12:30:...| null|2016-11-08 12:01:...|null|2016-11-04 10:15:...|
75| 75|false|[00 00 00 00 00 0...|[40 40 00 01 04 1...|false|null|11455|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...|[00 00 00 05 00]|[00 00 00 0E 01 4...|[00 00
00 05 00]|[00 00 00 09 01 4...|[00 00 00 0E 01 4...|[00 00 00 09 01 4...|[00 00
00 09 01 0...|[00 00 00 0D 01 0...|[00 00 00 16 01 0...|[00 00 00 05 00]|[00 00
00 05 00]|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1002|ACME|901112500|901143750|901162500|940112500|
32.79841232299805|-116.76612854003906|null|2016-07-20 14:40:...| 0| 1| 3|
0| null|false|false|2016-11-06 05:46:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 894.7491| null| 900.0002| null|
2|2016-11-08 11:46:...| null|2016-11-08 12:01:...|null|2016-11-08 12:31:...|
null| null| null|
null|2016-11-08 12:31:...| null| null|null|2016-11-06 05:46:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|12005|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1003|ACME|901112500|901143750|901162500|940112500| 32.79841232299805|
-116.7425308227539| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-06 21:48:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 899.9504| null| 900.0002| null|
2|2016-11-08 06:48:...| null| null|null|2016-11-08 12:33:...|
null| null| null|
null|2016-11-08 12:33:...| null| null|null|2016-11-06 21:48:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|10101|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1004|ACME|901112500|901143750|901162500|940112500|
32.79841232299805|-116.71894073486328| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-07 01:04:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 899.99994| null| 900.0002| null|
2|2016-11-08 01:34:...| null| null|null|2016-11-08 12:34:...|
null| null| null|
null|2016-11-08 12:34:...| null| null|null|2016-11-07 01:04:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|10101|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1005|ACME|901112500|901143750|901162500|940112500|
32.81824493408203|-116.78971862792969| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-06 18:06:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 889.1015|5731.9375|890.06525|5731.9375|
2|2016-11-08 10:36:...| null| null|null|2016-11-08 12:36:...|
null|2016-11-08 11:37:...| null|
null|2016-11-08 12:36:...| null|2016-11-08 11:37:...|null|2016-11-06 18:06:...|
75| 75|false|[00 00 00 00 00 0...|[40 00 01 00 41 0...|false|null|12056|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...|[00 00 00 05 00]|[00 00 00 0E 01 4...|[00 00
00 05 00]|[00 00 00 09 01 4...|[00 00 00 0E 01 4...|[00 00 00 09 01 4...|[00 00
00 09 01 0...|[00 00 00 0D 01 0...|[00 00 00 16 01 0...|[00 00 00 05 00]|[00 00
00 05 00]|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1006|ACME|901112500|901143750|901162500|940112500|
32.81824493408203|-116.76612091064453| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-07 21:07:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 895.9555| null| 900.0002| null|
2|2016-11-08 12:07:...| null|2016-11-08 11:37:...|null|2016-11-08 12:37:...|
null| null| null|
null|2016-11-08 12:37:...| null| null|null|2016-11-07 21:07:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|12047|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1007|ACME|901112500|901143750|901162500|940112500|
32.81824493408203|-116.74252319335938| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-06 07:54:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 898.3778| null| 900.0002| null|
2|2016-11-08 11:09:...| null| null|null|2016-11-08 12:39:...|
null| null| null|
null|2016-11-08 12:39:...| null| null|null|2016-11-06 07:54:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|10172|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1008|ACME|901112500|901143750|901162500|940112500|
32.81824493408203|-116.71892547607422| 107|2016-10-14 00:03:...| 0| 1| 3|
0|false|false|false|2016-11-06 19:55:...|null| 0|null|940112500| null|
null|75.000046|0.9999999| 899.2627| null| 900.0002| null|
2|2016-11-08 09:55:...| null| null|null|2016-11-08 12:25:...|
null| null| null|
null|2016-11-08 12:25:...| null| null|null|2016-11-06 19:55:...|
75| 75|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null|10105|
15| null| false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 55| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
|1009|ACME|901112500|901143750|901162500|940112500|32.838077545166016|-116.78972625732422|
107|2016-10-14 00:03:...| 0| 1| 3| 0|false|false|false|2016-11-06
22:27:...|null| 0|null|940112500| null| null|75.000046|0.9999999| 899.87164|
null| 900.0002| null| 2|2016-11-08 08:27:...| null|
null|null|2016-11-08 12:27:...| null| null|
null| null|2016-11-08 12:27:...| null|
null|null|2016-11-06 22:27:...| 75| 75|false|[00 00 00 00 00 0...|[00 00
00 00 00 0...|false|null|10111| 15| null| false| null|
null| null| 15|null|null| null| 1|null|null| null| null| null| null|
null| null| null| 55| null| 3|null| null|[00 00 00 00 00 0...|
null|[00 00 00 0E 01 4...| null| null|[00 00 00 0E
01 4...| null| null| null|[00 00
00 16 01 0...| null| null|[00 00 00 0E 01 0...|[00 00 00
0E 01 0...|
|1010|ACME|901112500|901143750|901162500|940112500|32.838077545166016|-116.76612091064453|
107|2016-10-14 00:03:...| 0| 1| 3| 0|false|false|false|2016-11-06
21:43:...|null| 0|null|940112500| null| null|75.000046|0.9999999| 899.9999|
null| 900.0002| null| 2|2016-11-08 01:13:...| null|
null|null|2016-11-08 12:28:...| null| null|
null| null|2016-11-08 12:28:...| null|
null|null|2016-11-06 21:43:...| 75| 75|false|[00 00 00 00 00 0...|[00 00
00 00 00 0...|false|null|10084| 15| null| false| null|
null| null| 15|null|null| null| 1|null|null| null| null| null| null|
null| null| null| 55| null| 3|null| null|[00 00 00 00 00 0...|
null|[00 00 00 0E 01 4...| null| null|[00 00 00 0E
01 4...| null| null| null|[00 00
00 16 01 0...| null| null|[00 00 00 0E 01 0...|[00 00 00
0E 01 0...|
|1011|ACME|901112500|901143750|901162500|940112500|
32.79841232299805|-116.78971862792969| 106|2016-10-14 00:03:...| 0| 0| 8|
0|false|false|false|2016-11-06 18:30:...|null| 0|null|940112500| null| null|
111.5431|0.9999999| 1471.827| null|1800.0004| null| 2|2016-11-07
05:30:...| null| null|null|2016-11-08 12:32:...|
null| null|2016-11-02 02:30:...|2016-11-02 02:30:...|2016-11-08
12:30:...| null| null|null|2016-11-06 18:30:...| 90|
90|false|[00 00 00 00 00 0...|[00 00 00 00 00 0...|false|null| 6040| 30| null|
false| null| null| null| 15|null|null| null|
1|null|null| null| null| null| null| null| null| null| 5| null|
3|null| null|[00 00 00 00 00 0...| null|[00 00 00 0E 01 4...|
null| null|[00 00 00 0E 01 4...| null|
null| null|[00 00 00 16 01 0...| null|
null|[00 00 00 0E 01 0...|[00 00 00 0E 01 0...|
+----+----+---------+---------+---------+---------+------------------+-------------------+----+--------------------+----+----+----+----+-----+-----+-----+--------------------+----+---+----+---------+-----+-----+---------+---------+----------+---------+---------+---------+---+--------------------+-----+--------------------+----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+-----+--------------------+----+--------------------+------+------+-----+--------------------+--------------------+-----+----+-----+----+-----+---------+--------------------+------+---------+----+----+----+-----+----+----+----+------+-----+-----+-----+-----+-----+---------+----+-------+----+----+-----+--------------------+----------------+--------------------+----------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+----------------+----------------+--------------------+--------------------+
only showing top 20 rows
10910 [ERROR] main com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery - sql
error:
org.apache.spark.sql.AnalysisException: cannot resolve '"I"."CI"' given input
columns: [RRB, NA2FR, RM, LBMT, IIICS, ATBMR, LMT, TSM, NA2UFTBFR, AGTBRE,
NGPD, TM, CS, LDTI, LTET, ATBRR, ARM, TD, LRMTB, VIB, SS13FSK34, CI, LSST, TMF,
FPT, LNG, NCG, FA, ATBM, RFD, TPMSM, VMR, LSRT, CWR, LPORET, AHDM, BLPST, VP,
ULHT, BAPS, UAS, LEDC, UUC, TPM, CSRE, NA2TLS, TMFM, LRMTR, FPR, NA2IT, BPR,
FR, LMTB, ATBR, ARTJ, RRN, BATBBR, LAT, LPET, LRMTP, MHDMS1, LRMT, FTB, BBRI,
FBM, LMTN, LTFCT, DT, LEMDMT, BLPT, BAS, LERMT, VMJ, NA2PT, HLB, RID, LICSCD,
RRR, LMTR, BLBR, BADUC, VE, MHDMS0, UAN, UTI, LAMT, LRMCT, NA28ARE, TC, LR,
SRM, TPMS];
at
org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:60)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:57)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:335)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:335)
at
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:69)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:334)
at
org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionUp$1(QueryPlan.scala:108)
at
org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$2(QueryPlan.scala:118)
at
org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$2$1.apply(QueryPlan.scala:122)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at
org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$2(QueryPlan.scala:122)
at
org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$2.apply(QueryPlan.scala:127)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
at
org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsUp(QueryPlan.scala:127)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:57)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:50)
at
org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:121)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:50)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:44)
at
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:34)
at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:133)
at
org.apache.spark.sql.DataFrame.org$apache$spark$sql$DataFrame$$withPlan(DataFrame.scala:2126)
at org.apache.spark.sql.DataFrame.select(DataFrame.scala:707)
at org.apache.spark.sql.DataFrame.select(DataFrame.scala:724)
at org.apache.spark.sql.DataFrame.select(DataFrame.scala:724)
at
com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery.testSpark(TestQuery.java:52)
at
com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery.mainTest(TestQuery.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
10952 [INFO] main org.apache.spark.storage.MemoryStore - Block broadcast_7
stored as values in memory (estimated size 379.2 KB, free 1709.5 KB)
10963 [INFO] main org.apache.spark.storage.MemoryStore - Block
broadcast_7_piece0 stored as bytes in memory (estimated size 27.3 KB, free
1736.7 KB)
10964 [INFO] dispatcher-event-loop-2 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_7_piece0 in memory on localhost:56810 (size: 27.3 KB, free:
5.1 GB)
10964 [INFO] main org.apache.spark.SparkContext - Created broadcast 7 from
newAPIHadoopRDD at PhoenixRDD.scala:46
11011 [INFO] main org.apache.phoenix.mapreduce.PhoenixInputFormat - Select
Statement: SELECT
"RID","I"."CI","I"."FA","I"."FPR","I"."FPT","I"."FR","I"."LAT","I"."LNG","I"."NCG","I"."NGPD","I"."VE","I"."VMJ","I"."VMR","I"."VP","I"."CSRE","I"."VIB","I"."IIICS","I"."LICSCD","I"."LEDC","I"."ARM","I"."FBM","I"."FTB","I"."NA2FR","I"."NA2PT","S"."AHDM","S"."ARTJ","S"."ATBM","S"."ATBMR","S"."ATBR","S"."ATBRR","S"."CS","S"."LAMT","S"."LTFCT","S"."LBMT","S"."LDTI","S"."LMT","S"."LMTN","S"."LMTR","S"."LPET","S"."LPORET","S"."LRMT","S"."LRMTP","S"."LRMTR","S"."LSRT","S"."LSST","S"."MHDMS0","S"."MHDMS1","S"."RFD","S"."RRN","S"."RRR","S"."TD","S"."TSM","S"."TC","S"."TPM","S"."LRMCT","S"."SS13FSK34","S"."LERMT","S"."LEMDMT","S"."AGTBRE","S"."SRM","S"."LTET","S"."TPMS","S"."TPMSM","S"."TM","S"."TMF","S"."TMFM","S"."NA2TLS","S"."NA2IT","S"."CWR","S"."BPR","S"."LR","S"."HLB","S"."NA2UFTBFR","S"."DT","S"."NA28ARE","S"."RM","S"."LMTB","S"."LRMTB","S"."RRB","P"."BADUC","P"."UAN","P"."BAPS","P"."BAS","P"."UAS","P"."BATBBR","P"."BBRI","P"."BLBR","P"."ULHT","P"."BLPST","P"."BLPT","P"."UTI","P"."UUC"
FROM TESTING.ENDPOINTS
11039 [INFO] main org.apache.spark.SparkContext - Starting job: count at
TestQuery.java:59
11039 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Registering RDD 33 (count at TestQuery.java:59)
11040 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Got job 3 (count at TestQuery.java:59) with 1 output partitions
11040 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Final stage: ResultStage 5 (count at TestQuery.java:59)
11040 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Parents of final stage: List(ShuffleMapStage 4)
11040 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Missing parents: List(ShuffleMapStage 4)
11041 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting ShuffleMapStage 4 (MapPartitionsRDD[33] at count at
TestQuery.java:59), which has no missing parents
11043 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_8 stored as values in memory (estimated size 18.0 KB, free
1754.8 KB)
11044 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_8_piece0 stored as bytes in memory (estimated size 8.1 KB, free
1762.9 KB)
11045 [INFO] dispatcher-event-loop-3 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_8_piece0 in memory on localhost:56810 (size: 8.1 KB, free:
5.1 GB)
11045 [INFO] dag-scheduler-event-loop org.apache.spark.SparkContext - Created
broadcast 8 from broadcast at DAGScheduler.scala:1006
11045 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting 1 missing tasks from ShuffleMapStage 4 (MapPartitionsRDD[33] at
count at TestQuery.java:59)
11045 [INFO] dag-scheduler-event-loop
org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 4.0 with 1 tasks
11046 [INFO] dispatcher-event-loop-4 org.apache.spark.scheduler.TaskSetManager
- Starting task 0.0 in stage 4.0 (TID 4, localhost, partition 0,PROCESS_LOCAL,
3864 bytes)
11046 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Running task 0.0 in stage 4.0 (TID 4)
11052 [INFO] Executor task launch worker-0 org.apache.spark.rdd.NewHadoopRDD
- Input split: org.apache.phoenix.mapreduce.PhoenixInputSplit@20b488
11683 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Finished task 0.0 in stage 4.0 (TID 4). 2500 bytes result sent to driver
11684 [INFO] task-result-getter-0 org.apache.spark.scheduler.TaskSetManager -
Finished task 0.0 in stage 4.0 (TID 4) in 638 ms on localhost (1/1)
11684 [INFO] task-result-getter-0 org.apache.spark.scheduler.TaskSchedulerImpl
- Removed TaskSet 4.0, whose tasks have all completed, from pool
11684 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- ShuffleMapStage 4 (count at TestQuery.java:59) finished in 0.638 s
11684 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- looking for newly runnable stages
11684 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- running: Set()
11684 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- waiting: Set(ResultStage 5)
11684 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- failed: Set()
11685 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting ResultStage 5 (MapPartitionsRDD[36] at count at
TestQuery.java:59), which has no missing parents
11686 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_9 stored as values in memory (estimated size 19.1 KB, free
1782.0 KB)
11687 [INFO] dag-scheduler-event-loop org.apache.spark.storage.MemoryStore -
Block broadcast_9_piece0 stored as bytes in memory (estimated size 8.6 KB, free
1790.7 KB)
11687 [INFO] dispatcher-event-loop-7 org.apache.spark.storage.BlockManagerInfo
- Added broadcast_9_piece0 in memory on localhost:56810 (size: 8.6 KB, free:
5.1 GB)
11687 [INFO] dag-scheduler-event-loop org.apache.spark.SparkContext - Created
broadcast 9 from broadcast at DAGScheduler.scala:1006
11688 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- Submitting 1 missing tasks from ResultStage 5 (MapPartitionsRDD[36] at count
at TestQuery.java:59)
11688 [INFO] dag-scheduler-event-loop
org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 5.0 with 1 tasks
11688 [INFO] dispatcher-event-loop-0 org.apache.spark.scheduler.TaskSetManager
- Starting task 0.0 in stage 5.0 (TID 5, localhost, partition 0,NODE_LOCAL,
1999 bytes)
11688 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Running task 0.0 in stage 5.0 (TID 5)
11691 [INFO] Executor task launch worker-0
org.apache.spark.storage.ShuffleBlockFetcherIterator - Getting 1 non-empty
blocks out of 1 blocks
11691 [INFO] Executor task launch worker-0
org.apache.spark.storage.ShuffleBlockFetcherIterator - Started 0 remote
fetches in 0 ms
11697 [INFO] Executor task launch worker-0 org.apache.spark.executor.Executor
- Finished task 0.0 in stage 5.0 (TID 5). 1830 bytes result sent to driver
11697 [INFO] task-result-getter-1 org.apache.spark.scheduler.TaskSetManager -
Finished task 0.0 in stage 5.0 (TID 5) in 9 ms on localhost (1/1)
11698 [INFO] task-result-getter-1 org.apache.spark.scheduler.TaskSchedulerImpl
- Removed TaskSet 5.0, whose tasks have all completed, from pool
11698 [INFO] dag-scheduler-event-loop org.apache.spark.scheduler.DAGScheduler
- ResultStage 5 (count at TestQuery.java:59) finished in 0.010 s
11698 [INFO] main org.apache.spark.scheduler.DAGScheduler - Job 3 finished:
count at TestQuery.java:59, took 0.658353 s
11699 [INFO] main com.sensus.NMSEngineOnHadoop.TuningEngine.TestQuery - Count
By phoenix spark plugin: 2834
11702 [INFO] pool-7-thread-1 org.apache.spark.SparkContext - Invoking stop()
from shutdown hook
11725 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/static/sql,null}
11726 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL1/execution/json,null}
11726 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL1/execution,null}
11726 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL1/json,null}
11726 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL1,null}
11726 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/static/sql,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL/execution/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL/execution,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/SQL,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/metrics/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/api,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/static,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/executors/threadDump/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/executors/threadDump,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/executors/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/executors,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/environment/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/environment,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/storage/rdd/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/storage/rdd,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/storage/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/storage,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/pool/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/pool,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/stage/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/stage,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages/json,null}
11727 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/stages,null}
11728 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/jobs/job/json,null}
11728 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/jobs/job,null}
11728 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/jobs/json,null}
11728 [INFO] pool-7-thread-1
org.spark-project.jetty.server.handler.ContextHandler - stopped
o.s.j.s.ServletContextHandler{/jobs,null}
11780 [INFO] pool-7-thread-1 org.apache.spark.ui.SparkUI - Stopped Spark web
UI at http://10.1.165.84:4040
11786 [INFO] dispatcher-event-loop-5
org.apache.spark.MapOutputTrackerMasterEndpoint -
MapOutputTrackerMasterEndpoint stopped!
11803 [INFO] pool-7-thread-1 org.apache.spark.storage.MemoryStore -
MemoryStore cleared
11804 [INFO] pool-7-thread-1 org.apache.spark.storage.BlockManager -
BlockManager stopped
11805 [INFO] pool-7-thread-1 org.apache.spark.storage.BlockManagerMaster -
BlockManagerMaster stopped
11807 [INFO] dispatcher-event-loop-2
org.apache.spark.scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint
- OutputCommitCoordinator stopped!
11812 [INFO] pool-7-thread-1 org.apache.spark.SparkContext - Successfully
stopped SparkContext
11812 [INFO] sparkDriverActorSystem-akka.actor.default-dispatcher-2
akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote
daemon.
11812 [INFO] pool-7-thread-1 org.apache.spark.util.ShutdownHookManager -
Shutdown hook called
11813 [INFO] pool-7-thread-1 org.apache.spark.util.ShutdownHookManager -
Deleting directory
C:\Users\xindian.long\AppData\Local\Temp\spark-4f0746ed-3d98-4f79-ab1d-571ae3a498c3
11814 [INFO] sparkDriverActorSystem-akka.actor.default-dispatcher-2
akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut
down; proceeding with flushing remote transports.
Process finished with exit code 0
