What is PuTTY?
PuTTY is (among other things) a free of SSH for Windows. I use PuTTy primarily as a method of communicating between my Windows workstation and Linux installations on VMs in VirtualBox.This article is part of the Hadoop Masterpage.
The purpose of SSH is to create a secure channel across an insecure network. SSH is an asymmetric encryption protocol. This means there are two keys. The public key will encrypt data, and can be disseminated to the world. The private key is used to decrypt data and must be kept hidden at all times.
Where do I download PuTTY?
I start at the official team site here:http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
And for this article, I’ve selected version 0.63.
PSCP is the putty version of scp which is a cp (copy) over ssh command.
Uploading Files
The use I will be describing in this article is to communicate from a Windows host device to a Linux Virtual Machine (VM), in this case, Ubuntu 14.04.Find the IP address on your Linux installation:
craigtrim@CVB:/usr/lib/apache/hadoop/2.5.2/bin$ ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:6f:e0:5d
inet addr:192.168.1.12 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2899 errors:0 dropped:0 overruns:0 frame:0
TX packets:1696 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:654302 (654.3 KB) TX bytes:600048 (600.0 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:815 errors:0 dropped:0 overruns:0 frame:0
TX packets:815 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:100433 (100.4 KB) TX bytes:100433 (100.4 KB)
In your windows command line, find the directory that the "pscp.exe" executable exists in.
The data I want to transfer exists in a temp directory on the c:/ drive of my Windows box, and I want to transfer this to a data sub-directory in my home directory in the Linux box.
I use this command to transfer all the files:
$ pscp c:\temp\* craigtrim@192.168.x.y:/home/craigtrim/data
The operational output is shown on the command line:
0106363.txt | 2 kB | 3.0 kB/s | ETA: 00:00:00 | 100%
0106364.txt | 3 kB | 3.2 kB/s | ETA: 00:00:00 | 100%
0106365.txt | 6 kB | 6.8 kB/s | ETA: 00:00:00 | 100%
0106366.txt | 3 kB | 3.9 kB/s | ETA: 00:00:00 | 100%
0106367.txt | 9 kB | 9.6 kB/s | ETA: 00:00:00 | 100%
0106368.txt | 3 kB | 3.9 kB/s | ETA: 00:00:00 | 100%
Downloading Files
Let's say I want to download the log files from my HDFS cluster on the NameNode.This is similar to the command above, with the exception of the remote system being placed first, and my local system being placed last:
$ pscp craigtrim@192.168.x.y:/usr/lib/apache/hadoop/2.5.2/logs/* c:/temp
Note that the local directory structure will need to exist first.
It is also possible to pass the username and password into the command:
$ pscp -l craigtrim -pw password 192.168.x.y:/usr/lib/apache/hadoop/2.5.2/logs/* c:\temp
hadoop-craigtrim-secondar | 39 kB | 39.1 kB/s | ETA: 00:00:00 | 100%
hadoop-craigtrim-namenode | 122 kB | 122.8 kB/s | ETA: 00:00:00 | 100%
yarn-craigtrim-resourcema | 36 kB | 36.1 kB/s | ETA: 00:00:00 | 100%
Authomation
I have also found it helpful to create a batch file in Windows with the command above.The path to the batch file can be placed in the PATH variable in System > Environment Variables. The command can also be parameterized. Once in the PATH, this command can be executed from any context in a given Windows Command session.
ReplyDeleteThanks for great posts on the subject - HADOOP
hadoop online training
Nice article....for more practice you may refer http://www.s4techno.com/lab-setup/
ReplyDeleteGreat article.. Explanation are very clear and step by step so easy to understand
ReplyDeletebig data training | hadoop training in chennai
Great and helpful blog to everyone.. Installation procedure are very clear and step by so easy to understand.. All installation commands are very clear and i learnt installation procedure easily form this blog so i install hadoop in my system very quickly.. thanks a lot for sharing this blog to us...
ReplyDeletebig data training and course syllabus | hadoop training topics | big data training topics
Excellent post. Very interesting to read. I really love to read such a nice post. Thanks! keep rocking. Big Data Hadoop Online Course Bangalore
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis information is really awesome thanks for sharing most valuable information.
ReplyDeleteworkday studio online training hyderabad
workday studio online training india
workday studio training india