gpt-3.5-turbo-0301· OpenAI
WildChat-4.8M (AI2)
scala spark多列怎么转多行
- User
scala spark多列怎么转多行
- Assistant
如果您已经有一张多列的Spark DataFrame,想要把其中多列转成多行,可以使用Spark DataFrame的
stack方法。比如,我们有这样一张含有多列的DataFrame:
val df = Seq( (1, "John", "Doe"), (2, "Jane", "Smith"), (3, "Bob", "Johnson") ).toDF("id", "first_name", "last_name")现在想要把
first_name和last_name这两列转成多行,可以使用stack方法:import org.apache.spark.sql.functions._ val stackedDf = df.selectExpr("id", "stack(2, 'first_name', first_name, 'last_name', last_name) as (col, val)") .select("id", "col", "val") stackedDf.show()这样就把
first_name和last_name这两列转成多行,每行包含id、列名(即col)和对应的值(即val)。输出结果:
+---+----------+------+ | id| col| val| +---+----------+------+ | 1|first_name| John| | 1| last_name| Doe| | 2|first_name| Jane| | 2| last_name| Smith| | 3|first_name| Bob| | 3| last_name|Johnson| +---+----------+------+
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.