Introduction to Stream Processing using Apache Spark

In my previous post, we looked at how Apache Spark can be used to ingest and aggregate the data using Spark SQL in a batch mode. There are different ways to create the Dataset from the raw data depending upon whether the schema of the ingested data is already well-known in advance (RDD of Java … Continue reading Introduction to Stream Processing using Apache Spark

Analyzing financial data with Apache Spark

With the rise of big data processing in the Enterprise world, it's quite evident that Apache Spark has become one of the most popular framework to process large amount of data to both in the batch mode and real-time. This article won't go into the overview of Apache Spark since there is already many good … Continue reading Analyzing financial data with Apache Spark