Financial Data Analysis – Kafka, Storm and Spark Streaming

In my earlier posts, we looked at how Spark Streaming can be used to process the streaming loan data and compute the aggregations using Spark SQL. We also looked at how the data can be stored in file system for future batch analysis. We discussed how Spark can be integrated with Kafka to ingest the … Continue reading Financial Data Analysis – Kafka, Storm and Spark Streaming

AWS Virtual Private Cloud

AWS Virtual Private Cloud (VPC) is a web service that allows provisioning of a logically isolated infrastructure in the public cloud with its own IP address range, subnets, internet gateway, ACLs and route table configuration. It can be thought of as an isolated data center in AWS. VPC does all the heavy lifting and makes it … Continue reading AWS Virtual Private Cloud

AWS – Simple Workflow Service

AWS Simple Workflow Service (SWF) is a web service that helps in coordinating the execution of automated tasks and human tasks within a workflow. The processing steps can be execution of scripts and code, web service calls, human action etc. SWF Workers and Deciders Workers are programs that interact with SWF service to get the tasks, … Continue reading AWS – Simple Workflow Service

AWS – Simple Notification Service

Message-oriented Middleware (MoM) supports a messaging type in which the messages can be broadcasted to multiple message consumers known as message subscribers. AWS Simple Notification Service is a type of messaging service in the cloud that is based on the pub-sub model. It allows the message publisher to send a message to a Topic which has multiple subscribers that … Continue reading AWS – Simple Notification Service

AWS – Simple Queue Service

Message-oriented Middleware (MoM) allows various applications and microservices within an enterprise to communicate with each other by sending messages asynchronously. Some of the MoMs traditionally used by enterprises are Websphere MQ, Active MQ, TIBCO, Rabbit MQ etc. Message Queue is used for point to point communication between the message producer and message consumer. AWS Simple … Continue reading AWS – Simple Queue Service

AWS – EC2, EBS and ELB

Elastic Compute Cloud (EC2) EC2 is a web service which provides re-sizable computing resources that could scale out or scale down very easily based on the varying work load. At a very basic level it's nothing but a virtual machine running as a guest on the host machine with the help of a hypervisor. Typically it … Continue reading AWS – EC2, EBS and ELB

AWS – Identity and Access Management

Identity and Access Management (IAM) is widely used in most of the enterprises to authenticate and authorize the users to grant access to applications and systems that supports various functions within the organization. It is one of the basic components when it comes to enterprise security and defense in-depth principles that organizations adopt to protect … Continue reading AWS – Identity and Access Management

Service Discovery – Spring Cloud Eureka

In microservices architecture, we tend to create lot of services which are loosely coupled from each other. Inherently this distributed system design has more inter-service communication and hence each service running in cloud should be able to locate other dependent services easily. Service discovery in the past has been done using service locators or configured … Continue reading Service Discovery – Spring Cloud Eureka

Spring Cloud Configuration

In my previous post on Cloud Native Applications, we discussed the benefits of 12 factor application and the characteristics that makes the application cloud ready. Once of the characteristics we discussed was to store the configuration outside the build artifact. Now let's see how we can leverage Spring Cloud Configuration server to store and inject the configuration that … Continue reading Spring Cloud Configuration

Cloud Native Applications

My previous post on Microservices discussed the benefits of decomposing monolithic applications into smaller services which could be deployed independently. We also discussed that microservices is an architectural style which promotes the applications to have a clear separation of concern a.k.a. bounded contexts and use the most appropriate technology stack. Once we embark on the microservices journey, we … Continue reading Cloud Native Applications