In the
i talked about how to create a custom UDF in scala for spark. But before you do that always check
Spark UDF's that are available with Spark already.
I have this sample Spark data frame with list of users
I wanted to sort the list of users in descending order of age so i used following 2 lines, first is to import functions that are available with Spark already and then i used
desc
function to order age in descending order
import org.apache.spark.sql.functions._
display(userDF.orderBy(desc("age")))
Now if i wanted to sort the data frame records using age in ascending order
display(userDF.orderBy(asc("age")))
This is sample of how to use the
sum()
function
userDF.select(sum("age")).show
1 comment:
The Wynn Hotel and Casino - Dr.CMD
lackluster casino and gaming experience. 광양 출장샵 서울특별 출장샵 경산 출장마사지 lackluster casino 인천광역 출장샵 and gaming experience. lackluster 상주 출장마사지 casino and gaming experience.
Post a Comment