In Hive queries, we can use Sort by, Order by, Cluster by, and Distribute by to manage the ordering and distribution of the output of a SELECT query. We will see this with an example. We have a table Employee in Hive, partitioned by Department. 0: jdbc:hive2://localhost:10000> desc employee; +————————–+———————–+———————–+–+…
We will see how to create a Hive table partitioned by multiple columns and how to import data into the table. Partitioning We can use partitioning feature of Hive to divide a table into different partitions. Each partition of a table is associated with a particular value(s) of partition column(s).…
We will see how to create a partitioned table in Hive and how to import data into the table. Partitioning We can use partitioning feature of Hive to divide a table into different partitions. Each partition of a table is associated with a particular value(s) of partition column(s). Partitioning allows…
We will see how to create a table in Hive using ORC format and how to import data into the table. ORC format ORC (Optimized Row Columnar) file format provides a highly efficient way to store Hive data. Using ORC format improves performance when reading, writing, and processing data in…
We will see how to create a table in Hive using SEQUENCEFILE format and how to import data into the table. Create table CREATE TABLE Employee( ID BIGINT, NAME STRING, AGE INT, SALARY BIGINT ) COMMENT ‘This is Employee table stored as sequencefile’ ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’…
We will see how to create a table in Hive using TEXTFILE format and how to import data into the table. TEXTFILE Textfile format stores data as plain text files. Textfile format enables rapid development due to its simplicity but other file formats like ORC are much better when it comes…
In this article we will see how to run Hive queries through JDBC. We are using apache-hive-1.0.1 and hiveserver2 is running on port 10000 on localhost. Jars Required To access Hive through JDBC we need to add the following jars in the classpath:- guava-18.0.jar hive-common-1.0.0.jar hive-exec-0.13.0.jar hive-jdbc-1.0.0.jar hive-metastore-1.0.0.jar hive-serde-1.0.0.jar hive-service-1.0.0.jar…
In this article we will see how to perform file operations on HDFS using Java APIs. Hadoop core jar is required to be added to the classpath. The class org.apache.hadoop.fs.FileSystem provide APIs for performing operations on HDFS. 1. Read a file from HDFS //Imports import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; //Code to…
We will see step by step process to install and configure a multi-node Hadoop 1.x setup. Cluster & User Suppose we want to setup a cluster of 5 machines. 192.168.0.1 192.168.0.2 192.168.0.3 192.168.0.4 192.168.0.5 We want to setup the cluster with the machine 192.168.0.1 as master node and all the…