Scaling Spring Batch Application on AWS with remote partitioning

In this post we'll look at how to scale a spring batch application on AWS using remote partitioning technique. Spring batch applications can be scaled by running multiple process in parallel on remote machines that can work independently on the partitioned data. There is a master step that knows how to partition the data and … Continue reading Scaling Spring Batch Application on AWS with remote partitioning

Running Batch Application using Spring Cloud Data Flow

In my previous article on running Spring Batch application in Cloud Foundry, we looked at how a Spring Batch application can be executed as a one-off task in PCF. We created a Spring boot batch application annotated as a Task and deployed it on PCF like any other cloud native application. Then the task was … Continue reading Running Batch Application using Spring Cloud Data Flow

Running Batch Application in Cloud Foundry

Cloud Foundry has become a platform of choice for large enterprises to run cloud native applications and transform the way applications are built and deployed. It provides the agility, scalability and flexibility to continuously launch new features quickly while reducing the risk. Cloud Foundry has been widely adopted to run Microservices which are associated with … Continue reading Running Batch Application in Cloud Foundry

Automated build using Jenkins Pipeline

In my earlier post on using Spark for financial analysis, we were either running the code within IntelliJ or doing a manual Gradle build. In this post, let's take a look at how we can leverage Jenkins pipeline for automated builds. The code discussed in this post is available in Github. Jenkins is a web … Continue reading Automated build using Jenkins Pipeline

Running Java application in Docker Container

In one of my earlier posts, we looked at how to create a Spark Application to read the Data from a CSV file. In this post, we'll take a look at how Docker image can be created for the Spark Financial Analysis application so that it can be easily run inside a container. The code … Continue reading Running Java application in Docker Container

Financial Data Analysis using Kafka, Storm and MariaDB

In my previous posts, we looked at how to integrate Kafka and Storm for streaming loan data and cleansing the data before ingesting it into processing pipeline for aggregating the data. We also looked at how to leverage Liquibase for managing the relational database in form of immutable scripts that could be version controlled. This fits … Continue reading Financial Data Analysis using Kafka, Storm and MariaDB

Microservices – Database management using Liquibase

Over the last few years, the proliferation of microservices and cloud native architecture patterns have surfaced new challenges that are resulting in new tools and techniques being adopted by enterprises. This allows for a seamless transition for enterprises in their cloud native journey to be more agile and nimble. In this post we'll look at … Continue reading Microservices – Database management using Liquibase