Use R to connect with HDFS and Hive

This document will provide some tips of R on how to connect Hive and HDFS with R. The RHive manual has a good document on how to do installation and initial setup for RHive before we start. In this document, I will provide more information on Hive and HDFS related stuff.

As we know the following command will connect HiveServerHost at port 10000 with HiverServer2:

> library(RHive)
> rhive.connect("HiveServerHost", 10000, hiveServer2)

If we give no arguments for rhive.connect, it will connect the Hive server with port number 10000 on local host:

> rhive.connect()

Then you can do query works on the Hive server by rhive.query(...), however, you need more power by using RHive, such as access HDFS and write a table onto it.

If you want to access a HDFS, you need to connect the name node of the HDFS first:

> rhive.hdfs.connect("hdfs://namenode:8020")

If above command doesn't work or not supported, add a dot(.) before the command, like:

> .rhive.hdfs.connect("hdfs://namenode:8020")


Next, we can try to create a 3-column table named "test" in Hive and save it in HDFS by using R: 

> L3 <- LETTERS[1:3]
> d <- data.frame(cbind(x=1, y=1:10), fac=sample(L3, 10, replace=TRUE))
> rhive.write.table(data=d, tableName="test")
[1] "test"

And show the content in table test:
> rhive.query("select * from test");
   x  y fac
1  1  1   B
2  1  2   A
3  1  3   A
4  1  4   B
5  1  5   B
6  1  6   A
7  1  7   C
8  1  8   A
9  1  9   B
10 1 10   A

You should check that there is a file located at /user/hive/warehouse/test/ in HDFS by the command:
> rhive.hdfs.ls("/user/hive/warehouse/test/")

Which means the content of the table has been saved to HDFS.

There are two good slides that you can refer on how to use RHive basic functions and functions related to HDFS:

RHive Basic Functions Tutorials
RHive HDFS Functions Tutorials

BricKiller is Now Released!

Now my first iOS game BricKiller has been released! Go ahead to get it on iTunes Store:

Game Logo

Snapshot While Playing

It is a game to play with bricks within a board, you can move one brick from one place to another place at one time,  and get the bricks exploded away from the board by putting some of them in the same color in a horizon line, vertical line, or diagonal line. More bricks that you can kill from the play board, more space in the board can place bricks. Once the bricks fill up the board, game will be over.

It looks like gobang or  tictactoe at the first glance, but it is much interesting for various colorful bricks and funny roles to make you win the game. Every time you move the brick, you need to give the maximum chance that the bricks can make a line easily without blocking the move for other bricks.

BricKiller means killing the bricks as many as you can. The more bricks you kill, the more space you can reserve on the board, the less chance that the brick will block your way, and the more bricks in the same color you have to make in a line.

Main Menu

Every time if you move bricks without bricks killed, more bricks will show up, we call it a wave of bricks. The number of bricks that are shown up in one wave in each stage is different. That makes one part to make difficulty different from stage to stage.

BricKiller have two modes to play: the normal mode and the endless mode. In normal mode, each stage has an initial set of bricks(a colorful diagram) that is waiting you to solve. Normal mode also have 3 worlds of stages. Each world have different criteria to eliminate the bricks. World I requires 3 bricks with the same color in a line to eliminate bricks, and World II and World III require 4 bricks and 5 bricks respectively. Typically, the stages in World III is more difficult than World I and World II.

This game support Game Center, you can get rankings with your score in Endless Mode and compete with other guys all over the world.

This game provides 100+ levels, come and take challenge to unlock them!