Rajesh AJ - BIGDATA

HOW TO INGEST THE DATA INTO ELASTICSEARCH AND VIEW IN KIBANA THROUGH HIVE Objective: The purpose of this tutorial is to walk you through the process of using Hive Table and push those datas to Elasticsearch and view this data in Kibana. Prerequisites: You should already have installed the Hortonworks Sandbox (HDP 2.5 Tech Preview). You should already have installed Elasticsearch and kibana. Scope: This tutorial was tested using the following environment and components: Ubuntu 14.04 HDP 2.4 Tech Preview on Hortonworks Sandbox Elasticsearch 2.4.0 Kibana 4.6.1 ES-Hadoop 5.1.1 jar Requirement: Steps followed to implement: 1.HIVE TABLE CREATION: 1.1 HIVE TABLE WITH LOCATION DATA 1.2 HIVE TABLE WITH PHYSICIAN DATA 1.3 HIVE TABLE WITH ONOCOLOGY DATA(MERGE THE TABLE 1.1 and 1.2) 2.INDEX FILE: 3.External table to hold the Hive data for ElasticSearch Index: 4.Ingest the Hive data to ElasticSearch: 1.HIVE TABLE CREATION: ~~~~~~~~~...

How to Load data from Twitter and push        data to Elasticsearch using NiFi & Creating a Kibana dashboard of Twitter data pushed to Elasticsearch with NiFi Article: This article shows you how to create a NiFi data flow using the GetTwitter and PutElasticsearch processors.. and  walk you through the process of creating a dashboard in Kibana using Twitter data pushed to Elasticsearch via NiFi. PART 1:[Load data from Twitter and push data to Elasticsearch using NiFi] Objective: The purpose of this tutorial is to walk you through the process of using NiFi to pull data from Twitter and push it to Elasticsearch and view those data in Kibana. Prerequisites: You should already have installed the Hortonworks Sandbox (HDP 2.5 Tech Preview). You should already have twitter access keys. You create your access keys here: Twitter Apps . Read more here: Twitter Docs . Scope: This tutorial was tested using the following envi...