Running Hive Queries Using Spark SQL
Running Hive Queries Using Spark SQL Hive queries using Spark SQL. This instructional blog post explores how it can be done. Prerequisite: Hadoop Cluster configured in your system Hive installed and configured with Hadoop Spark installed on the top of Hadoop eco-system Some more configurations need to be done after the successful configuration of Hadoop, Hive, and Spark. Open your spark-env.sh file which is present in the $SPARK_HOME/conf directory and open the spark-env.sh file. Here, add the HIVE_HOME as shown below. 1 export HIVE_HOME=/path_to_hive_installed_directory Now copy the hive-site.xml file present in the $ HIVE_HOME/conf directory to the $ SPARK_HOME/conf directory. In hive-site.xml at $ SPARK_HOME/conf directory Change hive.execution engine from tez to mr <property> <name> hive.execution.engine </name> <value> mr </value> ...