出错如下
9-07-2020 17:22:44 CST dataset_task INFO - 20/07/29 17:22:44 INFO dataaccumulation.CalculateDataSet$: sql == select cardinality(merge(cast(hll AS HyperLogLog))) as uv ,-1 as categoryid, -1 as brandid , -1 as dsid, -1 as campaignid ,idtype from dw_saas.rpt_dataasset_base_hll where idtype='phoneNumber' and pday <= 20200728 group by idtype 29-07-2020 17:22:44 CST dataset_task INFO - 20/07/29 17:22:44 INFO util.log: Logging initialized @38886ms 29-07-2020 17:22:46 CST dataset_task INFO - Exception in thread "main" java.sql.SQLException: Error executing query 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:54) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.util.PrestoUtil$.executeSql(PrestoUtil.scala:15) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.spark.saas.dataaccumulation.CalculateDataSet$.insertToMysql(CalculateDataSet.scala:59) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.spark.saas.dataaccumulation.CalculateDataSet$.assemblesql(CalculateDataSet.scala:122) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.spark.saas.dataaccumulation.CalculateDataSet$.execute(CalculateDataSet.scala:160) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.spark.saas.dataaccumulation.CalculateDataSet$.main(CalculateDataSet.scala:287) 29-07-2020 17:22:46 CST dataset_task INFO - at com.xxx.mip.data_assets.spark.saas.dataaccumulation.CalculateDataSet.main(CalculateDataSet.scala) 29-07-2020 17:22:46 CST dataset_task INFO - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 29-07-2020 17:22:46 CST dataset_task INFO - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 29-07-2020 17:22:46 CST dataset_task INFO - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 29-07-2020 17:22:46 CST dataset_task INFO - at java.lang.reflect.Method.invoke(Method.java:498) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) 29-07-2020 17:22:46 CST dataset_task INFO - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 29-07-2020 17:22:46 CST dataset_task INFO - Caused by: com.facebook.presto.jdbc.internal.airlift.http.client.RuntimeIOException: java.io.EOFException: HttpConnectionOverHTTP@6ef8c84e(l:/192.168.163.133:21156 <-> r:/192.168.163.133:9083,closed=false)[HttpChannelOverHTTP@2bfa5bf5(exchange=HttpExchange@4a8204e7 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@1a1b7558(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator{s=START}],recv=HttpReceiverOverHTTP@328974ed(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]] 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.airlift.http.client.ResponseHandlerUtils.propagate(ResponseHandlerUtils.java:20) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.airlift.http.client.FullJsonResponseHandler.handleException(FullJsonResponseHandler.java:55) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.airlift.http.client.FullJsonResponseHandler.handleException(FullJsonResponseHandler.java:35) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.airlift.http.client.jetty.JettyHttpClient.execute(JettyHttpClient.java:307) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.client.StatementClient.<init>(StatementClient.java:96) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.QueryExecutor.startQuery(QueryExecutor.java:63) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.PrestoConnection.startQuery(PrestoConnection.java:588) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:49) 29-07-2020 17:22:46 CST dataset_task INFO - ... 18 more 29-07-2020 17:22:46 CST dataset_task INFO - Caused by: java.io.EOFException: HttpConnectionOverHTTP@6ef8c84e(l:/192.168.163.133:21156 <-> r:/192.168.163.133:9083,closed=false)[HttpChannelOverHTTP@2bfa5bf5(exchange=HttpExchange@4a8204e7 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@1a1b7558(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator{s=START}],recv=HttpReceiverOverHTTP@328974ed(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]] 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:274) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.http.HttpParser.parseNext(HttpParser.java:1308) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:179) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:129) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:89) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:108) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) 29-07-2020 17:22:46 CST dataset_task INFO - at com.facebook.presto.jdbc.internal.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) 29-07-2020 17:22:46 CST dataset_task INFO - at java.lang.Thread.run(Thread.java:748) 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO spark.SparkContext: Invoking stop() from shutdown hook 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO server.AbstractConnector: Stopped Spark@619f2afc{HTTP/1.1,[http/1.1]}{0.0.0.0:4040} 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO ui.SparkUI: Stopped Spark web UI at http://mip-test-hdp133:4040 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices 29-07-2020 17:22:46 CST dataset_task INFO - (serviceOption=None, 29-07-2020 17:22:46 CST dataset_task INFO - services=List(), 29-07-2020 17:22:46 CST dataset_task INFO - started=false) 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO cluster.YarnClientSchedulerBackend: Stopped 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO memory.MemoryStore: MemoryStore cleared 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO storage.BlockManager: BlockManager stopped 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO storage.BlockManagerMaster: BlockManagerMaster stopped 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO spark.SparkContext: Successfully stopped SparkContext 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO util.ShutdownHookManager: Shutdown hook called 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO util.ShutdownHookManager: Deleting directory /data/hadoop/spark-2.4.3-bin-hadoop2.7/spark-5a4159bf-65c8-4573-9c56-ba2608f8eb64 29-07-2020 17:22:46 CST dataset_task INFO - 20/07/29 17:22:46 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-99eca7d4-7099-4f5f-a03c-dca471f2d785 29-07-2020 17:22:47 CST dataset_task INFO - Process completed unsuccessfully in 215 seconds. 29-07-2020 17:22:47 CST dataset_task ERROR - Job run failed! java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1 at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:304) at azkaban.execapp.JobRunner.runJob(JobRunner.java:786) at azkaban.execapp.JobRunner.doRun(JobRunner.java:601) at azkaban.execapp.JobRunner.run(JobRunner.java:562) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1 at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:125) at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:296) ... 8 more 29-07-2020 17:22:47 CST dataset_task ERROR - azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1 cause: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1 29-07-2020 17:22:47 CST dataset_task INFO - Finishing job dataset_task at 1596014567072 with status FAILED
解决办法：检查Presto url中的端口是否正确