tag:blogger.com,1999:blog-1906126944535337964.post5120367258477301672..comments2023-10-30T06:04:29.415+01:00Comments on Improve Yourself and Your Code: Functional Data Transformation with SparkPaweł Włodarskihttp://www.blogger.com/profile/04891037231290616803noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-1906126944535337964.post-8781976072023091492015-11-09T21:39:34.586+01:002015-11-09T21:39:34.586+01:00Hi, yes the photo is from tatra mountains :) And I...Hi, yes the photo is from tatra mountains :) And I chose DataFrames because it was easier for me to work with/transform tabular data in our project. So I decided here to use it as an practical illustration of functional approach to data transformation.Paweł Włodarskihttps://www.blogger.com/profile/04891037231290616803noreply@blogger.comtag:blogger.com,1999:blog-1906126944535337964.post-11893491188428980442015-11-02T09:15:55.165+01:002015-11-02T09:15:55.165+01:00really enjoyed reading this post, thank you for wr...really enjoyed reading this post, thank you for writing it. What motivated your choice of data frame versus RDD (eg, sequence of tuples from a case class)? Perhaps it's just your personal preference generally, but i'm interested to know if you chose Data Frame because you found it easier for building an FP pipeline (versus RDD). (also wonder whether the photo is of the tatras--i have been many times but never in summer)doughttps://www.blogger.com/profile/16041019983406055716noreply@blogger.com