This is a simple exercise and following are the steps for setting up a Maven project in eclipse:

• Create a new Maven project in Eclipse as shown below:
  • From package explorer view, goto New -> Other -> Maven -Select Maven project -> Fill in group id, artifact id, package name and click finish
  • You should now see a new Maven project which has a base pom.xml file.
  • Open the pom file and add following dependencies in it:
    • <!– Spark –>
      <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.10</artifactId>
      <version>1.6.1</version>
      </dependency>
      <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-sql_2.10</artifactId>
      <version>1.6.1</version>
      <scope>provided</scope>
      </dependency>
      <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-hive_2.10</artifactId>
      <version>1.6.1</version>
      <scope>provided</scope>
      </dependency>
  • First dependency will include Sparks core libraries, and other two are needed for exploring Spark SQL. We will explore Spark SQL in later posts, but this is the setup that will work for most of our examples.