val dataset = Seq(1).toDF
scala> dataset.explain(true)
== Parsed Logical Plan ==
LocalRelation [value#216]
== Analyzed Logical Plan ==
value: int
LocalRelation [value#216]
== Optimized Logical Plan ==
LocalRelation [value#216]
== Physical Plan ==
LocalTableScan [value#216]LocalRelation Logical Query Plan
LocalRelation is a leaf logical plan that allow functions like collect or take to be executed locally, i.e. without using Spark executors.
|
Note
|
When Dataset operators could be executed locally, the Dataset is considered local.
|
LocalRelation represents Datasets that were created from local collections using SparkSession.emptyDataset or SparkSession.createDataset methods and their derivatives like toDF.
It can only be constructed with the output attributes being all resolved.
The size of the objects (in statistics) is the sum of the default size of the attributes multiplied by the number of records.
When executed, LocalRelation is translated to LocalTableScanExec physical operator.