Apache Kafka – Producers and Consumers

This post will provide a quick overview on how to write Kafka Producer and Kafka Consumer with a Kafka broker running locally. First, let's set-up the Kafka broker locally by downloading the TAR file and running the required scripts. Other option is to run Kafka broker locally using Docker image, however I'll stick to the … Continue reading Apache Kafka – Producers and Consumers

Introduction to Stream Processing using Apache Spark

In my previous post, we looked at how Apache Spark can be used to ingest and aggregate the data using Spark SQL in a batch mode. There are different ways to create the Dataset from the raw data depending upon whether the schema of the ingested data is already well-known in advance (RDD of Java … Continue reading Introduction to Stream Processing using Apache Spark

Analyzing financial data with Apache Spark

With the rise of big data processing in the Enterprise world, it's quite evident that Apache Spark has become one of the most popular framework to process large amount of data to both in the batch mode and real-time. This article won't go into the overview of Apache Spark since there is already many good … Continue reading Analyzing financial data with Apache Spark

JWT – Token Based Authentication

In my earlier post on Cryptography, we looked at some of the cryptographic techniques and cryptographic functions that are commonly used to secure the data. In this post, we'll discuss JSON Web Token (JWT) which is one of the most commonly used token based authentication. It has become quite popular since it allows the distributed systems to … Continue reading JWT – Token Based Authentication

AWS – Relational Database Service

Amazon Relational Database Service (RDS) is a fully managed and cost efficient database service that makes it easy to provision, manage, and scale a relational database in the cloud. Amazon RDS provides an option to choose from the 6 available relational database engines - Commercial Oracle Microsoft SQL Server Open Source MySQL PostgreSQL MariaDB Cloud … Continue reading AWS – Relational Database Service

AWS S3 – Access Management

In my earlier posts on Cloud Storage and AWS S3, we discussed different storage types offered by the cloud providers and the Simple Storage Service (S3) by Amazon for storing the objects in the cloud. In this post, we'll look at how to manage the access to buckets and objects stored in the AWS S3. Buckets … Continue reading AWS S3 – Access Management

Cloud Storage Types – Object, Block and File

Cloud storage is making inroads and increasingly becoming quite common in enterprises these days due to the advantages it offers in terms of availability, durability and cost. Cloud storage solution can either be deployed in the private cloud or accessed over the internet in the public cloud depending upon the sensitivity of the data and compliance … Continue reading Cloud Storage Types – Object, Block and File

AWS DNS Service – Route 53

In one of earlier posts on DNS, we looked at the basic functionality provided by the DNS service and some of the important concepts related to the DNS protocol. AWS Route 53 is a distributed managed service that provides both the public and private DNS lookup service with a very high availability and scalability. It makes it … Continue reading AWS DNS Service – Route 53

AWS Elastic Load Balancer

A load balancer is a device that acts as a reverse proxy and distributes the application traffic across multiple servers. This results in increased capacity and greater reliability of the applications running behind the load balancer. Generally load balancers are grouped into 2 types Layer 4 load balancer - Acts on the data available in network … Continue reading AWS Elastic Load Balancer

Domain Name System – An Overview

Domain Name System (DNS) is a networking protocol that converts the human friendly domain name to an IP address. IP addresses (IPv4 or IPv6) uniquely identify the devices connected to the internet and helps in routing the network packets from the source to destination. DNS server can be thought as a directory that maintains the list of … Continue reading Domain Name System – An Overview