https://dataprd.com/tags/plan/ Recent content in Plan on Dataprd.Com Hugo en-us Wed, 12 Mar 2014 18:17:16 +0000 https://dataprd.com/posts/a-multi-tiered-big-data-warehouse-processing-facility/ Wed, 12 Mar 2014 18:17:16 +0000 https://dataprd.com/posts/a-multi-tiered-big-data-warehouse-processing-facility/ <p>The article details an exemplary setup for a multi-tiered data warehouse and processing facility using Hadoop and batch type of data analysis. Requirements in brief:</p> <ul> <li>High peaks on data acquisition (e.g. financial transactions of yearly festivals, R&amp;D facility with images or videos flowing in only at the time of experimenting) <ul> <li>Velocity is high at peak times, Gigabytes / second for short time (max. few hours / day)</li> <li>Data to be quickly accessed for data processing for some weeks after acquisition</li> </ul> </li> <li>High volume of data to be stored for long-time <ul> <li>Data to be accessed some years after acquisition, access can be delayed by days after request</li> </ul> </li> </ul> <p>The below setup details 3 tiers:</p>