scala> sql("SELECT explode(array(10,20))").explain
== Physical Plan ==
Generate explode([10,20]), false, false, [col#68]
+- Scan OneRowRelation[]
scala> sql("SELECT explode(array(10,20))").queryExecution.optimizedPlan.expressions(0)
res18: org.apache.spark.sql.catalyst.expressions.Expression = explode([10,20])
val arrayDF = Seq(Array(0,1)).toDF("array")
scala> arrayDF.withColumn("num", explode('array)).explain
== Physical Plan ==
Generate explode(array#93), true, false, [array#93, num#102]
+- LocalTableScan [array#93]Generator
Generator is an interface for Catalyst’s expressions that produce zero or more records given a single input record.
Generator is the foundation for ExplodeBase expressions.
Explode Generator Unary Expression
Explode is a unary expression that produces a sequence of records for each value in the array or map.
Explode is a result of executing explode function (in SQL and functions)
PosExplode
|
Caution
|
FIXME |
ExplodeBase Unary Expression
ExplodeBase is the base class for Explode and PosExplode.
ExplodeBase is UnaryExpression and Generator with CodegenFallback.