Skip to main content

Local Spark

This article provides information about Local spark support in Guzzle.

Local spark users will only use it for development, learning and lightweight purposes.

Local spark does not support various features like :

  1. ADLS
  2. DBFS
  3. DELTA
  4. HIVE

Guzzle provides below parameter and configs for local spark

PropertyDescriptionDefault ValueRequired
Number of executorsSpecify the number of spark executor on you want to run job2No
Driver memorySpecify the allocated driver memory for running jobs512 mNo
Executor memorySpecify the allocated executor memory for running jobs512 mNo
Driver classpathSpecify the driver classpath of jobsNoneNo
Executor classpathSpecify the executor classpathNoneNo
JarsPut the jar path with comma separatedNoneNo
Additional argumentsSpecify the spark arguments.
Ex: --conf spark.development=true

Interface for Local Spark Compute