Passing configuration options to Hadoop

Posted March 7th, 2013 in Razno by Metod

If you want to override Hadoop job configuration options, you can do so via command line -D key=value.

When I wanted to pass multiple configuration options including some special characters:

  1. hadoop jar MyJob.jar MyJob -D key=something,else -D key2=jdbc:mysql://host?user=u&password=p

I got some errors like:

  1. -bash: -D: command not found

The solution is to put values in quotes like this:

  1. hadoop jar MyJob.jar MyJob -D key="something,else" -D key2="jdbc:mysql://host?user=u&password=p"

Enjoy! :)

Code snippet 3

Posted January 15th, 2011 in Snippets by Metod

At the moment I am doing some java programming. And since being primarily a php guy, I was wondering how to easily return an array from a method like you can in php:

  1. public function returnArray() {
  2.     return array(1, 2, 3);
  3. }

And I found a way which is quite nice:

  1. public int[] returnArray() {
  2.     return new int[]{1, 2, 3};
  3. }