Everand Logo

Welcome to Everand!

  • Discover millions of ebooks, audiobooks, and so much more with a free trial

    Only $11.99/month after trial. Cancel anytime.

    Real-Time Big Data Analytics
    Real-Time Big Data Analytics
    Real-Time Big Data Analytics
    Ebook644 pages6 hours

    Real-Time Big Data Analytics

    By Shilpi and Gupta Sumit

    5/5

    ()

    Read preview

    About this ebook

    Design, process, and analyze large sets of complex data in real time

    About This Book

    - Get acquainted with transformations and database-level interactions, and ensure the reliability of messages processed using Storm
    - Implement strategies to solve the challenges of real-time data processing
    - Load datasets, build queries, and make recommendations using Spark SQL

    Who This Book Is For

    If you are a Big Data architect, developer, or a programmer who wants to develop applications/frameworks to implement real-time analytics using open source technologies, then this book is for you.

    What You Will Learn

    - Explore big data technologies and frameworks
    - Work through practical challenges and use cases of real-time analytics versus batch analytics
    - Develop real-word use cases for processing and analyzing data in real-time using the programming paradigm of Apache Storm
    - Handle and process real-time transactional data
    - Optimize and tune Apache Storm for varied workloads and production deployments
    - Process and stream data with Amazon Kinesis and Elastic MapReduce
    - Perform interactive and exploratory data analytics using Spark SQL
    - Develop common enterprise architectures/applications for real-time and batch analytics

    In Detail

    Enterprise has been striving hard to deal with the challenges of data arriving in real time or near real time.
    Although there are technologies such as Storm and Spark (and many more) that solve the challenges of real-time data, using the appropriate technology/framework for the right business use case is the key to success. This book provides you with the skills required to quickly design, implement and deploy your real-time analytics using real-world examples of big data use cases.
    From the beginning of the book, we will cover the basics of varied real-time data processing frameworks and technologies. We will discuss and explain the differences between batch and real-time processing in detail, and will also explore the techniques and programming concepts using Apache Storm.
    Moving on, we’ll familiarize you with “Amazon Kinesis” for real-time data processing on cloud. We will further develop your understanding of real-time analytics through a comprehensive review of Apache Spark along with the high-level architecture and the building blocks of a Spark program.
    You will learn how to transform your data, get an output from transformations, and persist your results using Spark RDDs, using an interface called Spark SQL to work with Spark.
    At the end of this book, we will introduce Spark Streaming, the streaming library of Spark, and will walk you through the emerging Lambda Architecture (LA), which provides a hybrid platform for big data processing by combining real-time and precomputed batch data to provide a near real-time view of incoming data.

    Style and approach

    This step-by-step is an easy-to-follow, detailed tutorial, filled with practical examples of basic and advanced features.
    Each topic is explained sequentially and supported by real-world examples and executable code snippets.
    LanguageEnglish
    PublisherPackt Publishing
    Release dateFeb 26, 2016
    ISBN9781784397401
    Real-Time Big Data Analytics

    Related to Real-Time Big Data Analytics

    Related ebooks

    Databases For You

    View More
    View More

    Related podcast episodes

    Related articles

    • Article

      Understanding ELT & ETL

      Apr 1, 2021

      Understanding ELT & ETL
      8 min read

    • Article

      AI As A Service

      Jul 9, 2020

      AI As A Service
      2 min read

    • Article

      Lag Is Killing Games

      Jan 11, 2022

      Lag Is Killing Games
      8 min read

    • Article

      Hot Picks

      Mar 9, 2021

      Hot Picks
      13 min read

    • Article

      Cloudy With No Chance Of Erp

      Nov 11, 2019

      ERP (enterprise resource planning) was born around the time the first ‘[Something] for Dummies’ book was published*. It’s typically inflexible, uncompromising software designed for large businesses, like banks, large corporations, manufacturing and s

      2 min read

    • Article

      Supercomputer On A Platter

      Apr 1, 2022

      CHENNAI-HEADQUARTERED automobile major TVS Motor Company uses high-performance computing (HPC) for running R&D simulations and testing the aero-dynamics of two-wheelers, which allows it to make the vehicles stable at speed and more efficient, cool en

      7 min read

    • Article

      Enterprise Soaring Success

      Aug 27, 2019

      Enterprise Soaring Success
      7 min read

    • Article

      News

      Feb 20, 2023

      Be careful who you buy from. Some Chinese crypto miners are doing everything they can to offload their heavily used mining GPUs now that they’ve nothing left to mine. They are going to interesting lengths to sell their inventory, like repainting them

      4 min read

    • Article

      From CPU To XPU

      Jun 11, 2021

      From CPU To XPU
      5 min read

    • Article

      News

      Feb 20, 2023

      Be careful who you buy from. Some Chinese crypto miners are doing everything they can to offload their heavily used mining GPUs now that they’ve nothing left to mine. They are going to interesting lengths to sell their inventory, like repainting the

      4 min read

    • Article

      HotPicks

      Nov 19, 2019

      HotPicks
      12 min read

    • Article

      Autodesk Maya 2020

      Mar 25, 2020

      Autodesk Maya 2020
      3 min read

    Related categories

    Reviews for Real-Time Big Data Analytics

    5/5

    1 rating0 reviews

    What did you think?

    Tap to rate

    Review must be at least 10 words

      Book preview

      Real-Time Big Data Analytics - Shilpi

      www.packtpub.com/authors.

      Customer support

      Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

      Downloading the example code

      You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

      You can download the code files by following these steps:

      Log in or register to our website using your e-mail address and password.

      Hover the mouse pointer on the SUPPORT tab at the top.

      Click on Code Downloads & Errata.

      Enter the name of the book in the Search box.

      Select the book for which you're looking to download the code files.

      Choose from the drop-down menu where you purchased this book from.

      Click on Code Download.

      Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

      WinRAR / 7-Zip for Windows

      Zipeg / iZip / UnRarX for Mac

      7-Zip / PeaZip for Linux

      Errata

      Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

      To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

      Piracy

      Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

      Please contact us at <copyright@packtpub.com> with a link to the suspected pirated material.

      We appreciate your help in protecting our authors and our ability to bring you valuable content.

      Questions

      If you have a problem with any aspect of this book, you can contact us at <questions@packtpub.com>, and we will do our best to address the problem.

      Chapter 1. Introducing the Big Data Technology Landscape and Analytics Platform

      The Big Data paradigm has emerged as one of the most powerful in next-generation data storage, management, and analytics. IT powerhouses have actually embraced the change and have accepted that it's here to stay.

      What arrived just as Hadoop, a storage and distributed processing platform, has really graduated and evolved. Today, we have whole panorama of various tools and technologies that specialize in various specific verticals of the Big Data space.

      In this chapter, you will become acquainted with the technology landscape of Big Data and analytics platforms. We will start by introducing the user to the infrastructure, the processing components, and the advent of Big Data. We will also discuss the needs and use cases for near real-time analysis.

      This chapter will cover the following points that will help you to understand the Big Data technology landscape:

      Infrastructure of Big Data

      Components of the Big Data ecosystem

      Analytics architecture

      Distributed batch processing

      Distributed databases (NoSQL)

      Real-time and stream processing

      Big Data – a phenomenon

      The phrase Big Data is not just a new buzzword, it's something that arrived slowly and captured the entire arena. The arrival of Hadoop and its alliance marked the end of the age for the long undefeated reign of traditional databases and warehouses.

      Today, we have a humongous amount of data all around us, in each and every sector of society and the economy; talk about any industry, it's sitting and generating loads of data—for instance, manufacturing, automobiles, finance, the energy sector, consumers, transportation, security, IT, and networks. The advent of Big Data as a field/domain/concept/theory/idea has made it possible to store, process, and analyze these large pools of data to get intelligent insight, and perform informed and calculated decisions. These decisions are driving the recommendations, growth, planning, and projections in all segments of the economy and that's why Big Data has taken the world by storm.

      If we look at the trends in the IT industry, there was an era when people were moving from manual computation to automated, computerized applications, then we ran into an era of enterprise level applications. This era gave birth to architectural flavors such as SAAS and PaaS. Now, we are into an era where we have a huge amount of data, which can be processed and analyzed in cost-effective ways. The world is moving towards open source to get the benefits of reduced license fees, data storage, and computation costs. It has really made it lucrative and affordable for all sectors and segments to harness the power of data. This is making Big Data synonymous with low cost, scalable, highly available, and reliable solutions that can churn huge amounts of data at incredible speed and generate intelligent

      Enjoying the preview?
      Page 1 of 1