hadoop&spark mapreduce对比 & 框架设计和理解

Hadoop MapReduce:


MapReduce在每次执行的时候都要从磁盘读数据,计算完毕后都要把数据放到磁盘


spark map reduce:







RDD is everything for dev:


Basic Concepts:



Graph RDD:

Spark Runtime:


schedule:


Depency Type:


Scheduler Optimizations:


Event Flow:


Submit Job:


New Job Instance:


Job In Detail:


executor.launchTask:


Standalone:




Work Flow:


Standalone detail:


Driver application to Clustor:


Worker Exception:


Executor Exception:


Master Exception:


Master HA:




版权声明:本文为博主原创文章,未经博主允许不得转载。

原文地址:https://www.cnblogs.com/stark-summer/p/4829824.html