Pages

Thursday, April 16, 2020

Hadoop WordCount tutorial

In this post we will write a word count program for Hadoop which is the equivalent of a Hello World program for any other language. This tutorial assumes that you have Hadoop already setup and is running. Begin by copying the code below and saving the file as WordCount.java
Note the name is important. It must be exactly equal to the class name. Before running the code set the environment variables as follows
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
export PATH=${JAVA_HOME}/bin:${PATH}
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

Compile WordCount.java and create a jar:
$ hadoop com.sun.tools.javac.Main WordCount.java
jar cf wc.jar WordCount*.class

Create two files for input to the Map-Reduce. We will make two files in the input folder as follows
mkdir input
echo "Hello World Bye World" > input/file01
echo "Hello Hadoop Goodbye Hadoop" > input/file02

We also need to create the input folder on HDFS
hadoop fs -mkdir -p /user/$USER/input

Now we need to move these files into HDFS
hadoop fs -copyFromLocal input/ /user/$USER/

Verify that the files have been copied
hadoop fs -ls /user/zaid/input

It should show two files as follows
Found 2 items
-rw-r--r--   1 zaid supergroup         22 2020-04-17 09:55 /user/zaid/input/file01
-rw-r--r--   1 zaid supergroup         28 2020-04-17 09:55 /user/zaid/input/file02

Now lets run the application
hadoop jar wc.jar WordCount /user/$USER/input /user/$USER/output

The program should run and show a lot of output and hopefully no errors. Once complete you can check the output as follows
$ hadoop fs -cat /user/$USER/output/part-r-00000

Monday, July 29, 2019

Gnosis multi signature wallet error - l@moz-extension://3ed71f6b-c0d8-4bfd-b8ef-8aa70dbe6eb0/background.js:1:1244922

When running the Gnosis wallet you might encounter a browser pop-up showing some cryptic error such as "0x98f057BA9F5c4c6849b550e4659B725A475285a1". Nothing to worry about, it simply means you have selected the wrong account when deploying the wallet. So simply change over to the account which you have setup as the owner and you will be good to go.

Monday, August 31, 2015

Android can not connect to adhoc network

Android does not seem to like adhoc networks and wont connect to one. But no need to fret you can make a simple network instead of an adhoc network and share your connection or do whatever you intend.

You can make a softAP on windows 7 by just typing in a couple of commands
in run type cmd, right click cmd.exe and select run as administrator

then type in

"netsh wlan set hostednetwork mode=allow ssid=YOURSSID key=YOURPASSPHRASE keyUsage=persistent"
replace YOURSSID and YOURPASSPHRASE with an SSID and passphrase of your choice. Remember the pass phrase must be thirteen characters long for WPA .

Example:
"netsh wlan set hostednetwork mode=allow ssid=zedAP key=abcdefghij keyUsage=persistent"

This is a one time step and will create the required “Microsoft Virtual WiFi Miniport adapter” and also set up your hostednetwork. You may now setup Internet Connection Sharing or Network Bridging for this newly found adapter. All this is onetime setting.

Now you need to start the hosted network by typing in

"netsh wlan start hostednetwork” 

And if you need to stop

"netsh wlan stop hostednetwork”

The connection is permanent but would need to restarted when PC is restarted, it is suggested to make a shortcut for the above command

On the empty space on desktop right click and select create shortcut
In the location type in
"C:\Windows\System32\cmd.exe /c netsh wlan start hostednetwork"
and hit next

Right click the shortcut created and goto properties->advanced and select run as administrator
Wola you are done!

But wait Android is not done with you yet. When you try to connect to the AP it may get stuck in an endless obtaining IP address loop. If that happens, select the network properties on your phone, goto advanced and select static instead of DHCP for the address. In the IP field observe the default gateway, assign a new IP same as the default gateway but change the last digit. If connecting more devices keep on incrementing by one.
Leave all other fields as is.


For instance if the default gateway is 192.168.1.1
use 192.168.1.2 as IP
and 192.168.1.3 and so on for subsequent devices.
Enjoy!!
 


Tuesday, February 17, 2015

Fuction keys for brightness not working in Ubuntu 14.04 and 16.04

Try adding the acpi_osi=\"!Windows 2012\" to the grub command line. You can do that by adding acpi_osi=\"!Windows 2012\" to the variable "GRUB_CMDLINE_LINUX_DEFAULT" in /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_osi=\"!Windows 2012\" Then run "sudo update-grub" Restart computer Enjoy!

Friday, August 22, 2014

Posting code to your blog

For those interested in posting code to their blog this link is a pure treat http://www.craftyfella.com/2010/01/syntax-highlighting-with-blogger-engine.html

Installing opencv in Ubuntu 12.04


sudo apt-get install libopencv-dev libcv-dev
then compile using
g++ `pkg-config --cflags opencv` codename.cpp `pkg-config --libs opencv` -o codename
if you get a highhui error replace
# include <highgui>
with
#include "opencv2/highgui/highgui.hpp"

Friday, May 2, 2014

Visual studio 2012 and windows 7 compatability issues

Upon installing visual studio 2012 in windows 7 you may notice that it gives compatibility warning on running visual studio. It fix is plain simple. Just go to the following link and download and install the patch. And get cracking :D
http://www.microsoft.com/en-pk/download/confirmation.aspx?id=36020