Analytical Performance in Data Lake Storage of Big Data Analytics by Databricks Delta Lake for Stock Market Analysis

n this paper, a Delta Lake workspace is created by using Azure portal; whereas, the ADLS Gen2 (ADLSG2) acts as primary storage account with a container to store workspace data. Despite the hype, ADLSG2 is immutable and it cannot perform analytics. This drawback leads to the introduction of Azure Databricks Delta Lake (ADDL) to fascinate the learning pattern that can be utilized for developing a support system to anlyze stock market and initiate better prediction on forecasted stock price. Databricks enhancement is an open source named Delta Lake, which remains as a pipeline for atomicity consistency isolation durability (ACID) table storage layer over cloud object stores. Finally, the ADDL performance is evaluated with the existing big data platform by using the parameters like memory usage and CPU usage.