Mastering RethinkDB

Table of Contents

Mastering RethinkDB

Credits

About the Author

About the Reviewer

www.PacktPub.com

Why subscribe?

Preface

What this book covers

What you need for this book 

Who this book is for 

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. The RethinkDB Architecture and Data Model

RethinkDB architectural components

Client drivers

RethinkDB query engine

RethinkDB clusters

Pushing changes to a RethinkDB client

Query execution in RethinkDB

Filesystem and data storage

About direct I/O

Data storage

Sharding and replication

Sharding in RethinkDB

Range-based sharding

Replication in RethinkDB

Indexing in RethinkDB

Automatic failover handling in RethinkDB

About voting replicas

The RethinkDB data model

RethinkDB data types

Binary objects

Geospatial queries in RethinkDB

Supported data types

RethinkDB model relationships

Embedded arrays

Merits of embedded arrays

Demerits of embedded arrays

Document linking in multiple tables

Merits of document linking

Demerits of document linking

Constraints and limitation in RethinkDB

Summary

2. RethinkDB Query Language

Embedding ReQL in a programming language

Performing CRUD operations using RethinkDB and Node

Creating new records

Reading the document data

Updating the document

Deleting the document

ReQL queries are chainable

ReQL queries are executed on a server

Performing conditional queries

Performing string operations

Performing MapReduce operations

Grouping the data

Counting the data

Sum

Avg

Min and Max

Distinct

Contains

Map and reduce

Calling HTTP APIs using ReQL

Handling binary objects

Performing JOINS

Accessing changefeed (real-time feed) in RethinkDB

Applications of changefeed

Performing geolocation operations

Storing a coordinate

Finding the distance between points

Performing administrative operations

Summary

3. Data Exploration Using RethinkDB

Generating mock data

Importing data in RethinkDB using HTTP

Importing data via file read

Executing data exploration use cases

Finding duplicate elements

Finding the list of countries

Finding the top 10 employees with the highest salary

Displaying employee records with a specific name and location

Finding employees living in Russia with a salary less than 50,000 dollars

Finding employees with a constant contact e-mail address

Finding employees who use class a C IP address

Summary

4. Performance Tuning in RethinkDB

Clustering

Creating and handling a RethinkDB cluster

Creating a RethinkDB cluster in the same machine

Creating a RethinkDB cluster using different machines

Creating a RethinkDB cluster in production

Securing our RethinkDB cluster

Using transport layer security

Binding the web administrative port

Executing ReQL queries in a cluster

Performing replication of tables in RethinkDB

Sharding the table to scale the database

Running a RethinkDB proxy node

Optimizing query performance

Summary

5. Administration and Troubleshooting Tasks in RethinkDB

Understanding access controls and permission in RethinkDB

RethinkDB user management

Failover handling in RethinkDB

Performing a manual and automatic backup in RethinkDB

Performing automatic backups

Restoring a RethinkDB database

Data import and export in RethinkDB

Importing data from MySQL to RethinkDB

Importing data from MongoDB to RethinkDB

Data migration to an updated version

Crash recovery in RethinkDB

Using third-party tools

ReQLPro

Chateau

Summary

6. RethinkDB Deployment

Deploying RethinkDB using PaaS services

Deploying RethinkDB on AWS

Deploying RethinkDB on Compose.io

Deploying RethinkDB on DigitalOcean

Deploying RethinkDB using Docker

The need for Docker

Installing Docker

Creating a Docker image

Deploying the Docker image

Deploying RethinkDB on a standalone server

Summary

7. Extending RethinkDB

Integrating RethinkDB with ElasticSearch

Introducing ElasticSearch

Installing ElasticSearch

Performing operations in ElasticSearch

The problem statement

Integration use cases

Search engine

Static website search

Integrating RethinkDB with RabbitMQ

Installing RabbitMQ

Developing producer code

Connecting to the RethinkDB instance

Creating a database and table if they do not exist

Connecting to RabbitMQ

Creating a channel and queue

Attaching a changefeed and integrating the RabbitMQ queue

Developing the consumer code

Connecting to RabbitMQ

Creating a channel and binding the queue

Understanding the RethinkDB protocol

Third-party libraries and tools

Summary

8. Full Stack Development with RethinkDB

Project structure

Data modeling for our application

Creating a Node.js server and routes

Integrating RethinkDB with Node.js

Integrating AngularJS in the frontend

Socket.io integration for message broadcasting

Summary

9. Polyglot Persistence Using RethinkDB

Introducing Polyglot Persistence

Using the RethinkDB changefeed as a Polyglot agent

Developing a proof-of-concept application with MongoDB and MySQL

Setting up the project

Developing a server using Express

Case - reading all users' data

Case - creating a new user

Case - updating user data

Case - deleting the user

Developing the Polyglot agent

Developing event consumers

Observer pattern

MySQL consumer

MongoDB consumer

Running the app

Further improvements

Integrating the message broker

Developing a distributed transaction

Summary

10. Using RethinkDB and Horizon

Workings of Horizon

Installing and configuring Horizon

Class Horizon

Connect()

Collection class

Method subscribe()

Watch method

CRUD methods

Developing a simple web application using Horizon

Setting up the project

Developing the JavaScript code

Developing the frontend

Horizon user management

Summary

Mastering RethinkDB

Mastering RethinkDB

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: December 2016

Production reference: 1131216

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham 

B3 2PB, UK.

ISBN 978-1-78646-107-0

www.packtpub.com

Credits

About the Author

Shahid Shaikh is an engineer, blogger, and author living in Mumbai, India. He is a full-time professional and a part-time blogger. He loves solving programming problems and he is, expert in software backend design and development.

Shahid has been blogging and teaching programming in practical way for more than two years on his blog. His blog is quite famous in the developer zone and people all around the world take advantage of his expertise in various programming problems related to backend development.

Shahid has also authored a book on Sails.js – MVC framework for Node.js published by Packt.

I would like to thank my parents, my family, and my friends for being kind and supportive during the book development period. I would like to thank my friends, who changed their plans according to my schedule for various occasions. I would also like to thank the RethinkDB team for helping me out with various architectural questions. You guys are awesome!

About the Reviewer

Rafael Ferreira dos Santos

Ted’s father, Geysla’s husband, Developer/Entrepreneur/Bjj addicted, 10 years working with software developer, loves to code, specially in ASP.NET and Node.js.

Thanks to Glenn Morton and the QuizJam team for such an amazing workplace. I would like to thank God and my wife for all the support and love that they give to me. I would not be in such an amazing moment without you.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Preface

RethinkDB is database built for real time web. It offers us variety of features over NoSQL databases already present in market such as very powerful query language, changefeed and easy scaling.

In this book, we are covering RethinkDB in deep and learning mastering level topics such as scaling, integration and deployment. We are also covering programming with RethinkDB along with step by step screenshot to help you understand the concepts easily. 

What this book covers

Chapter 1, The RethinkDB Architecture and Data Model, covers the architecture of RethinkDB and data modeling, along with revisiting the concepts of RethinkDB.

Chapter 2, RethinkDB Query Language, covers RethinkDB query language, or ReQL, which is the core and essential learning curve of RethinkDB. ReQL provides various SQL-like features such as join, indexing, and foreign keys, along with document-based storage with NoSQL.

Chapter 3, Data Exploration Using RethinkDB, covers data extraction and loading along with example use cases using ReQL.

Chapter 4, Performance Tuning in RethinkDB, covers various methods and tricks to improve the performance of RethinkDB.

Chapter 5, Administration and Troubleshooting Tasks in RethinkDB, covers failover mechanisms along with example use cases.

Chapter 6, RethinkDB Deployment, covers various options available to deploy RethinkDB on production.

Chapter 7, Extending RethinkDB. This chapter covers the integration of RethinkDB with other products, such as ElasticSearch.

Chapter 8, Full Stack Development with RethinkDB, covers the implementation of full stack JavaScript application using RethinkDB.

Chapter 9, Polyglot Persistence Using RethinkDB, covers complex synchronization application development using RethinkDB.

Chapter 10, Using RethinkDB and Horizon. This chapter covers the RethinkDB-powered framework called Horizon with a demo application.

What you need for this book 

A computer with at least 2 GB of RAM that can support Node.js and Java.

Who this book is for 

This book caters to all the real-time application developers looking forward to master their skills using RethinkDB. A basic understanding of RethinkDB and Node.js is essential to get the most out of this book. Developers working in backend development, full stack developers and database architect will find this book useful. 

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, path names, dummy URLs, user input and Twitter handles are shown as follows: We can include other contexts through the use of the include directive. 

A block of code is set as follows:

var hash = md5(email);

gravatarUrl = 'http://gravatar.com/avatar/' + hash + '?d=retro';

r.table('users').get(userId).update({

gravatar: r.http(gravatarUrl, {resultFormat: 'binary'})

}).run(conn, callback)

When it is necessary to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

r.table(users).reconfigure( {emergencyRepair: unsafe_rollback_or_erase} ).run(conn, callback);

Any command-line input or output is written as follows:

npm install --save express rethinkdb socket.io async body-parser

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes, for example, appear in the text like this: Clicking on the Next button moves you to the next screen.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to feedback@packtpub.com, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on https://www.packtpub.com/books/info/packt/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.

Hover the mouse pointer on the SUPPORT tab at the top.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box.

Select the book for which you're looking to download the code files.

Choose from the drop-down menu where you purchased this book from.

Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-RethinkDB. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

We also provide you a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from: https://www.packtpub.com/sites/default/files/downloads/MasteringRethinkDB_ColorImages.pdf.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at copyright@packtpub.com with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at questions@packtpub.com if you are having a problem with any aspect of the book, and we will do our best to address it.

Chapter 1. The RethinkDB Architecture and Data Model

RethinkDB is a real-time, open source distributed database. It stores JSON documents (basically unstructured data) in an operable format with distribution (sharding and replication). It also provides the real-time push of JSON data to the server, which redefines the entire real-time web application development.

In this chapter, we will look over its architecture and key elements in order to understand how RethinkDB supports these awesome features with high performance. We will also look over data modeling along with SQL operations in NoSQL, that is, joins.

Here is the list of topics we are going to cover in this chapter:

RethinkDB architectural components

Sharding and replication in RethinkDB

RethinkDB failover handling

The RethinkDB data model

Data modeling in RethinkDB

RethinkDB architectural components

RethinkDB's architecture consists of various components such as cluster, query execution engine, filesystem storage, push changes (real-time feed), and of course RethinkDB client drivers.

Refer to the following diagram to understand the block-level components of RethinkDB:

Client drivers

RethinkDB provides official client drivers for Node.js, Python, Ruby, and Java and various non official community drivers which are listed at the official website (https://rethinkdb.com/docs/install-drivers/). At the time of writing this book, only these languages were supported. In this book, we will refer to code examples with Node.js.

RethinkDB query engine

RethinkDB query handler, as name implies, performs query execution and returns the response to the client. It does so by performing lot of internal operations such as sorting, indexing, finding the cluster, or merging data from various clusters. All of these operations are performed by RethinkDB query handler. We will look at this in detail in the upcoming section.

RethinkDB clusters

RethinkDB is a distributed database designed for high-performance, real-time operations. RethinkDB manages distribution by clustering (sharding or replication). RethinkDB clusters are just another instance of the main process of RethinkDB and store data. We will look at sharding and replication in detail in the upcoming section.

Pushing changes to a RethinkDB client

This is a revolutionary concept introduced by RethinkDB. Consider this scenario: you are developing an application for the stock market where there are too many changes in a given amount of time. Obviously, we are storing every entry in the database and making sure that other connected nodes or clients know about these changes.

In order to do so, the conventional way is to keep looking (polling) for the data in the particular collection or table in order to find some changes. This improves the latency and turnaround time of packets, and we all know that a network call in a wide area network (WAN) is really costly. An HTTP call in a WAN is really costly.

Then came something called socket. In this, we do the polling operation but from the socket layer, not the HTTP layer. Here, the