Professional Documents
Culture Documents
What is it ? Architecture Related Projects Hive DDL Hive DML HiveQL Examples Business Intelligence
Hadoop What is it ?
A data warehouse for Hadoop Open source writen in Java Holds meta data in a relational database Allows SQL like queries Supports big data data sets Offers built in and user defined functions Has indexing
Hive Architecture
Where does Hive sit in the Hadoop architecture ?
Hive Architecture
Given an existing HDFS and Hadoop cluster Then add Hive and the meta data structure Use Flume and Sqoop to move data Use Hive LOAD DATA command to load from flat files Use ODBC for connectivity to your BI layer
Apache Flume move large data sets to Hadoop Apache Sqoop cmd line, move rdbms data to Hadoop Apache Hbase Non relational database Apache Pig analyse large data sets Apache Oozie work flow scheduler Apache Mahout machine learning and data mining Apache Hue Hadoop user interface Apache Zoo Keeper configuration / build
Hive - DDL
Create table
hive> CREATE TABLE customer (age INT, address STRING);
Partitions
hive> CREATE TABLE customer (age INT, address STRING) PARTITIONED BY ( sdate STRING) ;
Show table
hive> SHOW TABLES ;
Describe table
hive> DESCRIBE customer;
Hive - DDL
Alter table
hive> ALTER TABLE customer ADD COLUMNS ( age INT) ;
Drop table
hive> DROP TABLE customer;
Hive - DML
HiveQL Examples
selects all data from table for a partition but doesnt store it
hive> INSERT OVERWRITE DIRECTORY '/data/hdfs_file' SELECT a.* FROM customer a WHERE a.sdate='2008-08-15';
Use ODBC to connect Hive to your BI layer Now you can use BI tools like Business Objects
Create a universe over the Hive instance Create reports against the universe Create add hoc queries against the universe
Contact Us
www.semtech-solutions.co.nz info@semtech-solutions.co.nz
We offer IT project consultancy We are happy to hear about your problems You can just pay for those hours that you need To solve your problems