Passing configuration options to Hadoop

Posted March 7th, 2013 in Razno by Metod

If you want to override Hadoop job configuration options, you can do so via command line -D key=value.

When I wanted to pass multiple configuration options including some special characters:

hadoop jar MyJob.jar MyJob -D key=something,else -D key2=jdbc:mysql://host?user=u&password=p

I got some errors like:

-bash: -D: command not found

The solution is to put values in quotes like this:

hadoop jar MyJob.jar MyJob -D key="something,else" -D key2="jdbc:mysql://host?user=u&password=p"

Enjoy! 🙂

Code snippet 3

Posted January 15th, 2011 in Snippets by Metod

At the moment I am doing some java programming. And since being primarily a php guy, I was wondering how to easily return an array from a method like you can in php:

public function returnArray() {
    return array(1, 2, 3);

And I found a way which is quite nice:

public int[] returnArray() {
    return new int[]{1, 2, 3};