Answer by thebluephantom for why an action takes multiple jobs to get...
Specifically, pivot causes Spark to launch underwater jobs for the pivot values to be pivoted. You can supply them to obviate afaik, but that is not generally done.show and reading s3 paths also cause...
View Article
Image may be NSFW.
Clik here to view.
Clik here to view.
why an action takes multiple jobs to get completed in spark - scala
I am doing a pivot operation on top of a dataframe in spark-scala. But for the single pivot it is taking multiple jobs to get completed(as per the pic below)What could be the possible reason?This is...
View Article
More Pages to Explore .....
