Data Security on Dataprd.Com

Data Governance in the Big Data and Hadoop world - Whitepaper

Fri, 25 Aug 2017 16:11:15 +0000

My full whitepaper — Download PDF

Amidst vast data lakes and a high velocity of incoming data, enterprises are finding that despite the procedures, policies, and systems that are already in place, their data governance frameworks still aren’t delivering the insights needed to drive smart business decisions. Pain points such as privacy and security, appropriate permissions, and labor-intensive data mining processes are challenging general data governance frameworks that do not allow for flexibility or agility. Hadoop can be an excellent framework to support a large data storage repository with immense processing power. The key to success is understanding how to unleash Hadoop’s powerful storage capabilities. This white paper reveals:

Top 7 challenges of building a data lake

Sat, 07 Jan 2017 18:26:42 +0000

While from the technical perspective, deployment, management and provisioning tools are available to quickly set up a Hadoop cluster, introducing it to the organization is a tough task. Here are the biggest challenges of building a data lake 1. Understand the purpose and limitations of the technology

Hadoop environment provides a vendor independent tool-chain for data storage, management and analytics in a scalable fashion - it can manage infinite amount of data and with add-ons makes real-time and data science use-cases available for the enterprise
Many organizations do not have Big Data. They just load their RDBMS content and get surprised that a 20 node enterprise Hadoop setup’s performance is subpar compared with a single PostgreSQL instance. Sure, because the purpose and capabilities are different - we should compare apples with apples after all
In 95% of the cases stakeholders think that this new technology will substitute their painful proprietary data environment of RDBMSs. It is a highly distributed environment so the case is that it is never capable to substitute proprietary data systems. But it can live well beneath those providing extreme value. The purpose of the data lake is different. Ingest data, all of your data, and deal with portions of it without costly ETLs to move it from here to there. Refine your data system, be it processing, reporting or prediction models by slowly adding more datasets.

2. Security

Security of a Hadoop cluster

Fri, 14 Mar 2014 13:30:30 +0000

Hadoop is not only a data processing but a data warehouse solution. When we are talking about high amount of data we are talking about business value (the basis of the services) for a company. As Hadoop uses many nodes, network devices all the equipment must be included in a security plan - no weakest links are allowed. To secure our assets the following tools and approaches can be used.

Secure Linux based environment - communication for SMEs

Thu, 13 Mar 2014 13:11:16 +0000

Data governance procedures include the management of communication assets (e-mail, instant messaging, phone calls), which are considered as business secrets. While small enterprises should make calculations on using the cloud as the price benefit might be worth the negative impact on data management and ownership, a medium/large sized company is suggested to use own solutions with own security and data backup policies. Hereby - in brief - I present an exemplary cost-effective, secure setup for small to medium enterprises own infrastructure based on encrypted communication and messaging storage with data security and backup in mind. This is a high-level overview: data governance practices, backup policies, disaster recovery, update procedures, security testing, physical security, etc. are to be defined. Linux based servers: CentOS or Ubuntu Office 1: ~100-200 users