Data Analytics Tools on Dataprd.Com

Turning Natural Language Pricing into Structured Data for Automated Pricing

Sat, 15 Aug 2026 09:00:00 +0000

Every accommodation booking system eventually runs into the same wall. Owners describe their prices in prose, and software needs rows in a table.

What arrives looks like this, give or take a language. The colours show which sentence becomes which kind of row:

Owner's text

High season from 15 June to 22 August, 35,000 per night for up to 6 guests, extra person 5,000 per night.

Cleaning 15,000, free from 7 nights.

Christmas package 24 to 28 December, 4 nights 180,000.

Dogs 2,000 per night, maximum two, up to 15 kg.

Booking of 7 nights gets 1 night free.

Saturday to Saturday in high season.

Row	Captures
season	Date range, nightly rate, guests included, extra guest fee, cleaning fee and when it is waived
holiday	Fixed night count and a package total
pet fee	Per-animal rate, maximum animals, maximum weight
discount	Type `night_bonus`: 7 nights required, 1 night free
rule	Arrival and departure day, minimum stay

Six sentences become five rows, with every ambiguity resolved on the way. The model returns this:

Configure Apache Kylin with ODBC to work with MS PowerBI

Mon, 09 Jan 2017 21:13:16 +0000

PowerBI and Kylin - reporting from Hadoop via ODBC

This article discusses how to set up an ODBC interface for Kylin to work with Microsoft PowerBI. See previous article on the detailed dataset, environment setup, on what Kylin is and how to create a cube in Kylin. For the tutorial’s purposes, we will analyze the previously loaded flight delay data with Hadoop, Hive, HBase, Kylin, Kylin ODBC connector and MS PowerBI as an interface. With PowerBI, Microsoft provides a capable and simple BI tool for free (desktop version) - as competition heats up, this is a great strategy to gain some market share.

Evaluation of Apache Kylin 1.5.4.1 with HDP 2.5, performance comparison w Hive

Mon, 09 Jan 2017 20:24:45 +0000

Apache Kylin is a data cube solution on top of Hadoop providing an ODBC interface for BI tools. OLAP cubes boost performance for analytics via using a subset of data, enriched with pre-calculations on specific dimensions of interest. It enables loading dimensions from a Hive data source, therefore accelerating BI tool access via pre-calculating data and adding it to HBase. In our example we have a large dataset of flight information:

Create a Hadoop Cluster easily by using PXE boot, Kickstart, Puppet and Ambari to auto-deploy nodes

Fri, 08 Aug 2014 17:12:52 +0000

This tutorial is to showcase unattended and automatic install of multiple CentOS 6.5 x86_64 Hadoop nodes pre-configured with Ambari-agents and an Ambari-server host. After configuring automatic install of bare metal (No OS pre-installed) nodes, deploying a Hadoop cluster will be a matter of clicks. The setup uses:

PXE boot (for automatic OS install)
TFTP server (for PXE network install image)
Apache server (to serve the kickstart file for unattended install)
DHCP server (for assigning IP addresses for the nodes)
DNS server (for internal domain name resolution)
Puppet-master (for automatic configuration management of all hosts in the network, Ambari install included in Puppet manifests)
Ambari-master and agents (for managing Hadoop ecosystem deployment)

The setup assumes that the nodes are on the 192.168.0.0/255.255.255.0 network, the master is on 192.168.0.1 and its hostname is bigdata1.hdp The domain for the network server by the configuration is hdp and the clients are named as bigdata[1-254].hdp Download all files here (configuration files, PXEBoot Linux image, Kickstart file and custom script for adding a node on the master).

Analysis tutorial with Tableau Desktop

Sun, 11 May 2014 17:09:31 +0000

Tableau Desktop supports visual analysis and data discovery, converts the raw information to easy to understand graphical format with interactive charts. No coding is required to create rich visualization. Tableau Business Intelligence toolset has a Desktop, Server and Cloud version (none open-source products but as good as worth a post on the open-bigdata blog). In this post I check its Desktop evaluation version that lets us connect to many data sources (including Hadoop, MySQL, Excel, Text, …). I will use the same Weather.Csv as in the Hadoop analysis tutorial.

Environment setup for big data analytics

Tue, 04 Mar 2014 15:18:50 +0000

This article covers basic tools and technologies to use when conducting the first steps on big data analysis.

Linux as the base OS
- Debian
- Ubuntu
- RedHat
- CentOS
For basic data processing:
- Bash shell: environment for running multiple command-line Linux tools for data manipulation
  - Comes pre-installed with Linux; check Reference manual for usage
  - Bash might be not the default Linux shell, see how to switch to it
  - Learn Bash by examples
  - Most important Linux Commands and phenomena to master for data manipulation:
    - cat - output/concatenate a stream / file
    - head, tail - show the head or the tail of a stream / file
    - grep - to filter streams / lines of a file
    - sed - to manipulate streams / lines of a file
    - Regular expressions - used in a wide set of Linux tools
      - Explanation and tutorials
- AWK - simple data reformatter with compact coding features
  - The GNU Awk User’s Guide
  - AWK Tutorials
- Python - easy to learn, effective programming language with a huge amount of libraries available for various tasks. Great for data manipulation used from the command line.
  - Python Download
  - Python Documentation
  - Python Beginners Guide
  - Python Tutorials
And the big data analysis framework chosen based on the type of data analyzed. For the first step tutorials our suggestion would be:
- Hadoop, single cluster setup (can be downloaded pre-installed to a virtual appliance)
- Java based MapReduce programs
- Pig MapReduce query language