Enterprise Data Warehouse (EDW) is commonly found in large organizations that must deal with volumes of data. EDW platforms generally store historical data in a structured manner that allows Data and Analytic users to query and create insights. In this article we have outlined why maintaining an EDW platform is important. We have summarised the reasons why your platform may not be performing optimally.
At Stellar, Ulysses Villas, one of our BI consultants works on supporting many of our clients. One of his responsibilities is to make sure that the EDW overnight batch, runs successfully and completes within the Service Level Agreement (SLA). This is done as part of ongoing platform maintenance. Recently there were frequent Deadlock failures in the system (across multiple environments). It was important to identify the root cause with minimal disruption to the business.
Challenge
Ulysses had noticed that lately batch runs were taking longer than usual. Also, the volume of data had exponentially increased inside certain tables. The team at Stellar along with the client’s project team soon realised the cause was the increase in the volume of data coming from various sources and its increasing complexity due to newly implemented solutions. These instances caused delays in the completion of the overnight EDW batch run. It also delayed our client’s Data and BI analyst’s time to perform analysis and create insights on the different Power BI Reports and Reporting cubes.
Stellar’s solution
The team were able to formulate a solution to stabilize the Overnight EDW Batch runs. The next phase was testing the solution and it was critical as it affected the core packages of EDW. We needed to be sure that the proposed change to the system would not affect the volume and quality of the EDW data.
After weeks of testing, we successfully implemented the solution. The EDW has increased its stability across all the environments (TEST1, TEST2, Pre-Prod and Production).
If your organization is considering building an Enterprise Data Warehouse, talking to a Data and Analytics organization like Stellar Consulting Group could save you years of planning and testing. Stellar offers independent advice on technology that will suit your business. We also bring years of experience from having delivered numerous projects in the field of Data and Analytics.
