Pages

Thursday, April 16, 2020

Hadoop WordCount tutorial

In this post we will write a word count program for Hadoop which is the equivalent of a Hello World program for any other language. This tutorial assumes that you have Hadoop already setup and is running. Begin by copying the code below and saving the file as WordCount.java
Note the name is important. It must be exactly equal to the class name. Before running the code set the environment variables as follows
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
export PATH=${JAVA_HOME}/bin:${PATH}
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

Compile WordCount.java and create a jar:
$ hadoop com.sun.tools.javac.Main WordCount.java
jar cf wc.jar WordCount*.class

Create two files for input to the Map-Reduce. We will make two files in the input folder as follows
mkdir input
echo "Hello World Bye World" > input/file01
echo "Hello Hadoop Goodbye Hadoop" > input/file02

We also need to create the input folder on HDFS
hadoop fs -mkdir -p /user/$USER/input

Now we need to move these files into HDFS
hadoop fs -copyFromLocal input/ /user/$USER/

Verify that the files have been copied
hadoop fs -ls /user/zaid/input

It should show two files as follows
Found 2 items
-rw-r--r--   1 zaid supergroup         22 2020-04-17 09:55 /user/zaid/input/file01
-rw-r--r--   1 zaid supergroup         28 2020-04-17 09:55 /user/zaid/input/file02

Now lets run the application
hadoop jar wc.jar WordCount /user/$USER/input /user/$USER/output

The program should run and show a lot of output and hopefully no errors. Once complete you can check the output as follows
$ hadoop fs -cat /user/$USER/output/part-r-00000

Monday, July 29, 2019

Gnosis multi signature wallet error - l@moz-extension://3ed71f6b-c0d8-4bfd-b8ef-8aa70dbe6eb0/background.js:1:1244922

When running the Gnosis wallet you might encounter a browser pop-up showing some cryptic error such as "0x98f057BA9F5c4c6849b550e4659B725A475285a1". Nothing to worry about, it simply means you have selected the wrong account when deploying the wallet. So simply change over to the account which you have setup as the owner and you will be good to go.

Monday, August 31, 2015

Android can not connect to adhoc network

Android does not seem to like adhoc networks and wont connect to one. But no need to fret you can make a simple network instead of an adhoc network and share your connection or do whatever you intend.

You can make a softAP on windows 7 by just typing in a couple of commands
in run type cmd, right click cmd.exe and select run as administrator

then type in

"netsh wlan set hostednetwork mode=allow ssid=YOURSSID key=YOURPASSPHRASE keyUsage=persistent"
replace YOURSSID and YOURPASSPHRASE with an SSID and passphrase of your choice. Remember the pass phrase must be thirteen characters long for WPA .

Example:
"netsh wlan set hostednetwork mode=allow ssid=zedAP key=abcdefghij keyUsage=persistent"

This is a one time step and will create the required “Microsoft Virtual WiFi Miniport adapter” and also set up your hostednetwork. You may now setup Internet Connection Sharing or Network Bridging for this newly found adapter. All this is onetime setting.

Now you need to start the hosted network by typing in

"netsh wlan start hostednetwork” 

And if you need to stop

"netsh wlan stop hostednetwork”

The connection is permanent but would need to restarted when PC is restarted, it is suggested to make a shortcut for the above command

On the empty space on desktop right click and select create shortcut
In the location type in
"C:\Windows\System32\cmd.exe /c netsh wlan start hostednetwork"
and hit next

Right click the shortcut created and goto properties->advanced and select run as administrator
Wola you are done!

But wait Android is not done with you yet. When you try to connect to the AP it may get stuck in an endless obtaining IP address loop. If that happens, select the network properties on your phone, goto advanced and select static instead of DHCP for the address. In the IP field observe the default gateway, assign a new IP same as the default gateway but change the last digit. If connecting more devices keep on incrementing by one.
Leave all other fields as is.


For instance if the default gateway is 192.168.1.1
use 192.168.1.2 as IP
and 192.168.1.3 and so on for subsequent devices.
Enjoy!!
 


Tuesday, February 17, 2015

Fuction keys for brightness not working in Ubuntu 14.04 and 16.04

Try adding the acpi_osi=\"!Windows 2012\" to the grub command line. You can do that by adding acpi_osi=\"!Windows 2012\" to the variable "GRUB_CMDLINE_LINUX_DEFAULT" in /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_osi=\"!Windows 2012\" Then run "sudo update-grub" Restart computer Enjoy!

Friday, August 22, 2014

Posting code to your blog

For those interested in posting code to their blog this link is a pure treat http://www.craftyfella.com/2010/01/syntax-highlighting-with-blogger-engine.html

Installing opencv in Ubuntu 12.04


sudo apt-get install libopencv-dev libcv-dev
then compile using
g++ `pkg-config --cflags opencv` codename.cpp `pkg-config --libs opencv` -o codename
if you get a highhui error replace
# include <highgui>
with
#include "opencv2/highgui/highgui.hpp"

Friday, May 2, 2014

Visual studio 2012 and windows 7 compatability issues

Upon installing visual studio 2012 in windows 7 you may notice that it gives compatibility warning on running visual studio. It fix is plain simple. Just go to the following link and download and install the patch. And get cracking :D
http://www.microsoft.com/en-pk/download/confirmation.aspx?id=36020

Friday, April 4, 2014

Purple screen on windows start when dual booting with ubuntu

Recently i installed Ubuntu alongside windows 7 on my Dell 15R 5537. Initially everything worked fine but after some update perhaps; windows stopped working. On selecting windows from the boot menu, it would show a purple screen, however i could hear windows boot and its start screen. Googled for days but did not find an exact solution. Here is what I did

First try doing this
On the grub menu highlight the windows menu item using the keyboard and then press 'e' to the end of the line that begins with "set root= *******" add 'quiet splash' so that it becomes
"set root= ******* quiet splash"
now press ctrl + x
Does it solve the problem? If it did now you proceed to make the change permanent

Start ubuntu
press ctrl + alt + t to open up terminal
type 'sudo gedit'
in the editor navigate to /boot/grub/grub.cfg and open the file
In the file select the menu entry for windows and copy it
Now open the file /etc/grub.d/40_custom
Paste the menu entry that you copied previously after the comments on a new line
In the menu entry add the "quiet splash" as done before
Save the file
Now run
sudo update-grub
Reboot, enjoy!

Sunday, January 13, 2013

Saturday, January 12, 2013

Missing power option in Ubuntu 12.04

There is a feature/bug in Ubuntu 12.04 that does not let you choose actions on critical battery. I finally found a solution to the problem
gsettings set org.gnome.settings-daemon.plugins.power critical-battery-action 'nothing'
As it turns out the option can be set if you use the non-GUI method. For those who wish that the laptop suspends instead of hibernate (defult action). You should use
gsettings set org.gnome.settings-daemon.plugins.power critical-battery-action 'suspend'
NOTE: sudo is not required

Saturday, December 15, 2012

How to make a bootable Windows USB under any LINUX version

There are a lots of softwares available for making bootable and live USB's but they all fail at one distro or another. So here is a magic bullet that fits all. All you need is a bootable OR live distro's iso image file. If you have a live distro you'd get a live USB, if you only have bootable iso image you would get a bootable USB only. First locate your USB
sudo fdisk -l

Then run this, replace path-to-iso with the path to where the iso file is on your disk and the ? with drive letter for your USB
dd if=/path-to-iso of=/dev/sd?
sync

Dont forget the sync command at the end to flush the usb-cache. Example
dd if=/home/me/ubu.iso of=/dev/sdc
sync
Note: Do not include the numeric number e.g. its /dev/sdc and NOT /dev/sdc1!

Sunday, October 7, 2012

Generating random numbers in c++


#include "stdafx.h"
#include <iostream>
#include <time.h>
using namespace std;

int main()
{
 int c;
 srand(time(NULL));
        while(1)
        {
       cout<<rand();
        }
 return 0;
}

Lets have an in depth look at the code. The actual random numbers are generated by the rand() function. The numbers that it generates are not random but are pseudorandom i.e. they appear to be random but are not. In actual the rand() function generates a predictable series of numbers. The series however can be initialized with a seed value which in the above code is being done using the srand() function. The srand() needs only be called once during a typical program. However given a same seed value the program will generate the same random numbers every time the program is run. In order to avoid this repetition across program executions. We have passed the time(NULL) as an argument to the srand() function. Which will initialize the srand with a new value every time the function is called, hence now the numbers generated by rand() would appear to be random.

Saturday, October 6, 2012

Using the arrow keys in c++ using scan codes

As a programmer you may have been tempted to use the arrow keys to take input from the user, did you get frustrated and gave up because it was not as trivial as it should be?
Well it is a piece of cake but to eat it you have to earn it.
Wait no further....
For those in a hurry here is the code

#include <iostream>
#include <conio.h>
using namespace std;
int main()
{
 int c;
 while (1)
 {
  c = getch(); // first call gets the 0xE0 byte or decimal 224
  switch(c)    // lets just waste the value
  {
  case 0:
  case 224:
   switch (getch()) //Here the actual code for the key is got
   {
   case 72: cout << "up arrow\n"; break;
   case 75: cout << "left arrow\n"; break;
   case 77: cout << "right arrow\n"; break;
   case 80: cout << "down arrow\n"; break;
   default: cout << "extended key " << c << "\n";
   }
  }
 }
 return 0;
}


Lets De-magicify

And here is the treat for the geeks.
Lets examine the code step by step. Before diving into details lets have a look at the background. Any key pressed from the key board generates a unique code (think of it as a number stored as a HEX value), this code is known as Scancode Generally each code is one byte in size but some keys (keys with special functions) generate two or more bytes and this code is referred to as extended Scancodes

The following keys fall under the extended category
  • Arrow keypad
  • Home keypad
  • Right Alt
  • Right Ctrl
  • keyboards with the arrow pad, home pad and number pad
Since the arrow keys fall under the extended scancode category. Lets see what code is generated when an arrow key is pressed.

If an arrow key is pressed it generates two bytes, one the code for the key pressed prefixed with an 0xE0 or decimal 224 (scan code for left shift release) on most systems. The corresponding scan codes for the keys are
  • Up 72
  • Down 80
  • Left 75
  • Right 77

Scancodes c++ and the getch function

The getch function gets a single character from the console, since extended scan codes, especially the arrow keys generate two bytes, you can already judge why we'd need two calls to getch(), the first call gets the code 224 and the next one gets the actual code for the key pressed. The rest is plain c++ switch statements. Enjoy your programming :)

Thursday, July 5, 2012

How to configure a LAMP server on Ubuntu 10.04

A LAMP server refers to Linux, Apache, MySQL, PHP or Python. Assuming you have already got a Ubuntu 10.04 (LINUX) machine up and running lets first install an Apache server.
sudo apt-get install apache2

Before we proceed with setting up apache lets set up the hosts file, this file holds ip mappings to machine names.
sudo gedit /etc/hosts

You will see something like this
127.0.0.1	localhost.localdomain	localhost

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts


Add the line 127.0.0.1 test-site-name.local to make it look something like this, i have used the name "ccl" in this example
127.0.0.1	localhost.localdomain	localhost
127.0.0.1 	ccl.local		z3d-laptop

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts


Now when you are done setting up the hosts file, we'll now configure virtual hosting so that we can host multiple domains (or subdomains) with the server.
cd /etc/apache2/sites-available

Once in the directory lets make setup our virtual host, for this we'll use the template provided by apache. This file is named default. Create a copy of the file first.
sudo cp default your-site-name

Now open the file you just created (by copying) for editing
sudo gedit your-site-name

This should open something similar to this
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
<VirtualHost *:80>
    ServerAdmin webmaster@localhost

    DocumentRoot /var/www
    <Directory />
        Options FollowSymLinks
        AllowOverride None
    </Directory>
    <Directory /var/www/>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
    </Directory>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>

    ErrorLog /var/log/apache2/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog /var/log/apache2/access.log combined

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

</VirtualHost>

 

In this file add the line ServerName test-site-name.local just above the DocumentRoot directive (in front of line 4). Edit DocumentRoot /var/www path on line 4 and set it to /path-to-the-test-site-WITHOUT-trailing-slash. It should look something like this
DocumentRoot /path-to-the-test-site-WITHOUT-trailing-slash

Edit path on line 9 and set it to /path-to-the-test-site-WITH-trailing-slash/. It should look something like this
 Directory /path-to-the-test-site-WITH-trailing-slash/ 

Please Note: Be careful regarding where there is a trailing slash where there is none. Also change
AllowOverride None

to
AllowOverride All

for the first two directory nodes (the / one and the one with the path to your site). That will allow all the .htaccess files to work properly and allow redirection.
Now we are almost done, we just need to enable the site
sudo a2ensite test-site-name 
sudo /etc/init.d/apache2 restart

In the next step lets configure and install MySQL
apt-get install mysql-server 

when prompted for a password choose one and save it for future reference Note the configuration file for MySQL is located at /etc/mysql/my.cnf we will not be modifying it but you should know it for future reference. Running the next command is recommended for securing your server, especially if it is a production server. When you will run the command you will be prompted for many things answering them with a yes is recommended.
mysql_secure_installation 

Next lets create a database and grant control to a user
mysql -u root -p 
create database test-db;
grant all on test-db.* to 'zed' identified by 'password';
flush privileges;
quit

Note: The semicolons are important and identify end of MySQL commands. In the above case a database with the name of test-db has been created and the user zed has been granted full control over it. The password for the user is password. You can keep any password that you like. With Apache and MySQL installed we are now ready to move on to installing PHP. Install php
apt-get install php5 php-pear

If you are not experienced with PHP the following configurations are useful for more detailed logging and better performance
sudo gedit /etc/php5/apache2/php.ini

Change the fields as shown below
max_execution_time = 30
memory_limit = 64M
error_reporting = E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR
display_errors = Off
log_errors = On
error_log = /var/log/php.log
register_globals = Off

Now restart Apache
sudo /etc/init.d/apache2 restart

if you need MySQL support in PHP then install the following package
apt-get install php5-mysql
apt-get install php5-suhosin

The second package is a recommended package for security, it provided additionally security for PHP Restart Apache to make sure everything is correct
sudo /etc/init.d/apache2 restart

Wola! Your lamp server is now configured and good to go! :)

Thursday, May 24, 2012

Creating a Bootable USB the hassle free way

There are a lots of softwares available for making bootable and live USB's but they all fail at one distro or another. So here is a magic bullet that fits all. All you need is a bootable OR live distro's iso image file. If you have a live distro you'd get a live USB, if you only have bootable iso image you would get a bootable USB only. First locate your USB
sudo fdisk -l

Then run this, replace path-to-iso with the path to where the iso file is on your disk and the ? with drive letter for your USB
dd if=/path-to-iso of=/dev/sd?

Example
dd if=/home/me/ubu.iso of=/dev/sdc
Note: Do not include the numeric number e.g. its /dev/sdc and NOT /dev/sdc1!

Monday, February 13, 2012

Installing ns-2.35 on Ubuntu 10.04

A lot of people dread having to install ns2. Well here is a simple tutorial to get you through in just a single go. Mostly people fail at installing ns2 in the first go because of the missing libraries and dependencies.

Here is what you do to install ns2 in the first go.

1. Download ns-allinone-2.35

2. cd into the directory where you have downloaded the ns-allinone-2.35.tar.gz file.
tar -xvzf ns-allinone-2.35.tar.gz

3. Next you need to install the required libraries

sudo apt-get install autoconf automake gcc g++ build-essential libxmu-dev libtool libxt-dev 

6. Now cd into the ns-allinone-2.35 directory and do ./install , hopefully everything will go smoothly and you ns-2 will be installed. But before you can proceed you need to set the environment variables.

gedit ~/.bashrc
copy the following lines at the end of the file, be sure to change /your/path to the path where ns is installed on your system.

# LD_LIBRARY_PATH
OTCL_LIB=/your/path/ns-allinone-2.35/otcl-1.14
NS2_LIB=/your/path/ns-allinone-2.35/lib
X11_LIB=/usr/X11R6/lib
USR_LOCAL_LIB=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OTCL_LIB:$NS2_LIB:$X11_LIB:$USR_LOCAL_LIB

# TCL_LIBRARY
TCL_LIB=/your/path/ns-allinone-2.35/tcl8.5.10/library
USR_LIB=/usr/lib
export TCL_LIBRARY=$TCL_LIB:$USR_LIB

# PATH
XGRAPH=/your/path/ns-allinone-2.35/bin:/your/path/ns-allinone-2.35/tcl8.5.10/unix:/your/path/ns-allinone-2.35/tk8.5.10/unix
NS=/your/path/ns-allinone-2.35/ns-2.35/
NAM=/your/path/ns-allinone-2.35/nam-1.15/
PATH=$PATH:$XGRAPH:$NS:$NAM

for instance on my system i would replace /your/path with /home/z3d

9. source ~/.bashrc to have the changes take effect or you can simply reopen terminal.

10. Now cd into the folder /ns-allinone-2.35/ns-2.35/ and type the command ./validate
it will take a considerably longer time than it did to install and in almost an hour you should get a message all tests successful.

11. Now you can type ns and begin doing your work :)

Monday, January 9, 2012

Wednesday, November 2, 2011

How to check your ISO file for consistency before buring it to disc or using it

When you download a large file from the Internet there are chances that the file might be corrupted during download. In order to be sure that the download is not corrupt, you can always check its md5sum.
In case of Ubuntu you can check the md5 sum of the downloaded ISO using
md5sum ubuntu-11.10-desktop-i386.iso
Then you can compare this with the md5sum available on the website, from where you downloaded the ISO.

Tuesday, November 1, 2011

Windows not loading from grub menu, what to do??

Sometimes it may happen that after reinstalling grub the windows does appear in the grub menu but does not load. In case you are having such a problem. Boot up your Ubuntu machine
sudo su
update-grub