Secure Linux based environment - communication for SMEs

Thu, 13 Mar 2014 13:11:16 +0000

Data governance procedures include the management of communication assets (e-mail, instant messaging, phone calls), which are considered as business secrets. While small enterprises should make calculations on using the cloud as the price benefit might be worth the negative impact on data management and ownership, a medium/large sized company is suggested to use own solutions with own security and data backup policies. Hereby - in brief - I present an exemplary cost-effective, secure setup for small to medium enterprises own infrastructure based on encrypted communication and messaging storage with data security and backup in mind. This is a high-level overview: data governance practices, backup policies, disaster recovery, update procedures, security testing, physical security, etc. are to be defined. Linux based servers: CentOS or Ubuntu Office 1: ~100-200 users

Environment setup for big data analytics

Tue, 04 Mar 2014 15:18:50 +0000

This article covers basic tools and technologies to use when conducting the first steps on big data analysis.

Linux as the base OS
- Debian
- Ubuntu
- RedHat
- CentOS
For basic data processing:
- Bash shell: environment for running multiple command-line Linux tools for data manipulation
  - Comes pre-installed with Linux; check Reference manual for usage
  - Bash might be not the default Linux shell, see how to switch to it
  - Learn Bash by examples
  - Most important Linux Commands and phenomena to master for data manipulation:
    - cat - output/concatenate a stream / file
    - head, tail - show the head or the tail of a stream / file
    - grep - to filter streams / lines of a file
    - sed - to manipulate streams / lines of a file
    - Regular expressions - used in a wide set of Linux tools
      - Explanation and tutorials
- AWK - simple data reformatter with compact coding features
  - The GNU Awk User’s Guide
  - AWK Tutorials
- Python - easy to learn, effective programming language with a huge amount of libraries available for various tasks. Great for data manipulation used from the command line.
  - Python Download
  - Python Documentation
  - Python Beginners Guide
  - Python Tutorials
And the big data analysis framework chosen based on the type of data analyzed. For the first step tutorials our suggestion would be:
- Hadoop, single cluster setup (can be downloaded pre-installed to a virtual appliance)
- Java based MapReduce programs
- Pig MapReduce query language

Linux on Dataprd.Com

Secure Linux based environment - communication for SMEs

Environment setup for big data analytics