codedecoder

breaking into the unknown…


Leave a comment

ssh to remote server

we will ssh to server represented by system B from the local machine represented by system A. The detail of the two system is as below

system A :
name :  arun@192.168.173.54
username: arun
password: axyz

system B :
name :  kapil@192.168.173.53
username: kapil
password: axyz

PREREQUISITE :

1 -> ssh client and server should be installed on both the machine. Install them with below command

$sudo apt-get install openssh-client
$sudo apt-get install openssh-server

2 -> modifying ssh-server configuration on server machine i,e system B for us

Though , default works fine, It is better to look at the file to see what going on. login to system B
$sudo gedit /etc/ssh/sshd_config # it will open the configuration file for you

Below, are the some line, I felt worth describing

Port 22 # this is the default setting, when some one ssh to your system, they listen to this port by default. You can change it to some other value, which will make your server more secure as while doing ssh the person also need to pass the port no.

PubkeyAuthentication yes # If you do not want anyone to login directly with valid ssh key, but only with password set it to no

PermitRootLogin yes #set it to no, if you do not want to give root access to ssh login

after making the changes you need to start the ssh server with the below command

$sudo /etc/init.d/ssh restart

NOTE : You better backup this file before making any changes. if you make some mistake in syntex or some other things , sshd server will refuses to start due to an incorrect configuration directive. In worst case, If you have access to server only through ssh, then you will not able to login to server to correct the error i,e your server get locked for you after restart, if you make error in sshd_config file.

3-> ssh key should be available on bot the machine. run below command to generate them
$ssh-keygen -t dsa # keep pressing enter when asked for password etc
By default the public key is saved in the file ~/.ssh/id_dsa.pub, while ~/.ssh/id_dsa is the private key
LOGIN TO SERVER WITH SSH
we will ssh to our server(i,e system B)  from system A
-> Login with password

arun@arun-yadav:~$ ssh kapil@192.168.173.53
kapil@192.168.173.53’s password: kxyz
Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-31-generic-pae i686)
* Documentation:  https://help.ubuntu.com/
155 packages can be updated.
60 updates are security updates.
Last login: Wed Oct 31 14:14:49 2012 from arun-yadav.local
kapil@f3-MS-7636:~$ # so you are loged in to server

-> Login directly with ssh

for this you need to create a  authorized_keys file on ssh folder of server i,e /.ssh/authorized_keys and append your id_dsa.pub key content to it. provided, authorized_keys file exist in ssh folder of server(create it if it is not there and give it 600 permission) you can copy your id_dsa.pub to it with the below command from your local machine i,e the system A

$ ssh-copy-id kapil@192.168.173.53
kapil@192.168.173.53’s password: # you will get the below message
Now try logging into the machine, with “ssh ‘kapil@192.168.173.53′”, and check in:
~/.ssh/authorized_keys
to make sure we haven’t added extra keys that you weren’t expecting.
So, now when you do ssh, you will be directly loged in as you can see below
arun@arun-yadav:~$ ssh kapil@192.168.173.53
Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-31-generic-pae i686)
* Documentation:  https://help.ubuntu.com/
155 packages can be updated.
60 updates are security updates.
Last login: Wed Oct 31 14:18:48 2012 from arun-yadav.local
kapil@f3-MS-7636:~$   #so you are directly loged in to the server


Leave a comment

ssh Connection refused port 22

Recently, I have to automate my rails deployment. I want to first replicate my deployment script on my friend machine. So nothing should go wrong in the real server. So, I tried to ssh to my friend machine whose address is suresh@192.168.173.51, but welcomed by the error message.

$ ssh suresh@192.168.173.51
ssh: connect to host 192.168.173.51 port 22: Connection refused

I thought,  the target machine is not accessible on my network, but I was wrong, when I pinged the machine it responded me back

$ ping 192.168.173.51

64 bytes from 192.168.173.51: icmp_req=1 ttl=64 time=0.105 ms
64 bytes from 192.168.173.51: icmp_req=2 ttl=64 time=0.123 ms

while , searching on google, I come across ssh -v ip command, which tell information about ssh at given IP.

$ ssh -v 192.168.173.51 #but it also end with connection refused error
ssh: connect to host 192.168.173.51 port 22: Connection refused.

Some one suggested the possibility of duplicate Ip address on network. I doubt this, however to make this sure that two people are not assigned same IP by mistake on my network. I installed arp-scan. It is used to track all the assigned IP on your network

$ sudo apt-get install arp-scan
$ sudo su –
# arp-scan -I eth0 -l | grep 192.168.173.51 # it will list all system using the IP 192.168.173.51
192.168.173.51    00:e0:4c:02:c1:ef    REALTEK SEMICONDUCTOR CORP.

So,my assumption is wrong as the arp-scan showing only one assigned IP. Still do not know the reason y the error is coming.I tried to ssh to my own system as below…which is getting connected

$ ssh -v 192.168.173.54
OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.173.54 [192.168.173.54] port 22.
debug1: Connection established.

I looked, at my own system and my friend system and found that I have both ssh client and ssh server installed on my system but my friend system having only ssh client. I installed ssh server as below and it solved my problem

$sudo apt-get install openssh-server

So, This is the summary of things you should check if get connection refused error.

-> check that Internet is up on the system you are trying to ssh
-> check that ssh machine listing to default port 22. If not, say it listing to 3456, then while doing ssh pass the port option also as below
$ ssh suresh@192.168.173.51 -p 3456
If you want to see other available options, type below command
$ssh –help #it will list all the available options for you
-> check that firball not blocking the connection
-> check that ssh-client and ssh server is installed on ssh machine. If not, install them as below, you can get detail here .

$sudo apt-get install openssh-client

$sudo apt-get install openssh-server


					
		
	


Leave a comment

`block in tours’: undefined method `sub’

I was trying to use tourbus for load testing, when I got this error while running the test

$ tourbus -c 2 -n 3 thirdpillar_load_testing

/home/arun/.rvm/gems/ruby-1.9.3-p194@threepiller/gems/tourbus-2.0.1/lib/tourist.rb:79:in `block in tours’: undefined method `sub’ for :tour_homepage:Symbol (NoMethodError)

I looked at the line where the error is coming, it look like as below

 def tours
    methods.grep(/^tour_/).map {|m| m.sub(/^tour_/,”)}
 end

This code worked for ruby 1.87 but not for ruby 1.9.1 and above. I thnik in new ruby version methods are treated as symbol rather then string. So, basically we are applying the sub method of string to symbol. This problem can be solved if you convert the methods to string before applying the sub method as below.

  def tours
    methods.map(&:to_s).grep(/^tour_/).map {|m| m.sub(/^tour_/,”)}
  end

But. You should not change gem file yourself. I checked the tourbus git repository. I am surprised, to see that there do exist a issue posted for this problem, and it is solved also 4 months back, but the code is not pushed to rubgem.org i,e the default repository from where bundle install the gem. So I decided to pass exact git path to the tourbus gem in the gem file as below

gem ‘tourbus’, :git => ‘https://github.com/dbrady/tourbus.git’

Now,  on running bundle update, I find that , I got the correct code, but it is not installed at desired location (/gems folder of my gemset)where all gems are installed but instead to /bundler/gems folder of my gemset. Anyway, I decided to run the test again

$ tourbus -c 2 -n 3 thirdpillar_load_testing

But, now I getting many missing dependency errors. It is obvious as it is installed at some other folder and all other depending gems are installed at some other folder. I also felt that, the gem is not well maintained and documented so LET IT GO … I moved to other options 🙂


1 Comment

rvm not found or installed

RVM installation instruction is properly documented at the RVM site. The detail installation instruction are available here

Just run below commands on terminal and you are done.

$ sudo apt-get install bash

$ sudo apt-get install git

$ sudo apt-get install curl

$  \curl -L https://get.rvm.io | bash -s stable  #there is a backslash before curl. This prevents misbehaving if you have aliased it with configuration in your ~/.curlrc file.

……log message ending with as below….

Installation of RVM in /home/arun/.rvm/ is almost complete:

* To start using RVM you need to run `source /home/arun/.rvm/scripts/rvm`
in all your open shell windows, in rare cases you need to reopen all shell windows.

The last few line of instruction is very important. It tells where your rvm is installed and how to start it.

To test That your rvm is installed and working properly. run below command on the terminal

$ type rvm | head -1

If you get the message rvm is a function , it means its is working properly. But if you encountered message like rvm not found or rvm not installed. It just means that your rvm is not loaded to system path. check it out yourself

$ echo $PATH # for me it gives below result, you can see that my rvm is loaded to path variable

/home/arun/.rvm/gems/ruby-1.9.3-p194/bin:/home/arun/.rvm/gems/ruby-1.9.3-p194@global/bin:/home/arun/.rvm/rubies/ruby-1.9.3-p194/bin:/home/arun/.rvm/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

If you not found, rvm in your path variable you need to load it. All the things to do that is present in $HOME/.rvm/scripts/rvm file assuming that my rvm is installed in home folder, for me $HOME means /home/arun. You just need to source this file. You can do it in following ways

-> whenever you open the terminal run below command

$ source “$HOME/.rvm/scripts/rvm”

-> Create a .bash_profile file in home folder if not already exist and add below line to it

[[ -s “$HOME/.rvm/scripts/rvm” ]] && source “$HOME/.rvm/scripts/rvm”

Now, from terminal run the command

$source .bash_profile

This , step is almost same as last one. In both case, you have to run the source command for all the new terminal you open

-> add below line to .bashrc file

[[ -s “$HOME/.rvm/scripts/rvm” ]] && source “$HOME/.rvm/scripts/rvm”

This is the best option. As .bashrc file is automatically exicuted when you open a terminal, rvm will be automatically loaded when ever you open a terminal

-> add below line to /etc/profile

[[ -s “$HOME/.rvm/scripts/rvm” ]] && source “$HOME/.rvm/scripts/rvm”

This, will add rvm to $PATH variable, when ever you start your system. Though, this also server my purpose as I don not have to source the path every time. but I avoid it as, I do not want to load the rvm untill I really want to use it. But in production, I think this would be best as it will load the rvm when server get started

Removing RVM

Sometime, installation goes wrong and the best option is to uninstall and reinstall the things. Lets assume that, our last RVM installation was not proper. So, you can remove it completely with below command

$rvm implode # you will see the below message
Are you SURE you wish for rvm to implode?
This will recursively remove /home/arun/.rvm and other rvm traces?
(anything other than ‘yes’ will cancel) > yes # thus write yes and press enter
Removing rvm-shipped binaries (rvm-prompt, rvm, rvm-sudo rvm-shell and rvm-auto-ruby)
Removing rvm wrappers in /home/arun/.rvm/bin
Hai! Removing /home/arun/.rvm
/home/arun/.rvm has been removed.
rvm has been fully removed. Note you may need to manually remove /etc/rvmrc and ~/.rvmrc if they exist still


Leave a comment

load testing with rails

One of our project is extensively based on API calls. The API service providers have there own bottleneck and take a lot of time to process our request and send the response. The overall page load time is not encouraging. The client wanted to test the robustness  of application and to see the overall response time when the number of concurrent user increases. Here, come the need to do load testing for the application. I tried to evaluate the tools available in the market. I came across below options.

Ruby Gems

being ruby developer, first I tried to find out some tools on ruby community like ruby toolbox and rubygems . I found some options at this link.

I give a try to two of the gems tourbus and trample.  They didn’t suits my need. I looked into other gems also and felt below

-> There is still not any decent gem available on rails community to do a average load testing

-> Most of the gem owners have left development and maintenance of the gems they started building

-> None, of the gems have a proper documentation or a demonstrable examples

-> The two gems I have tried have problem in there installation itself. Strange thing is that tourbus has fixed the reported installation issue with ruby 1.9.3 and above 4 months back, but still not pushed the code to rubygem.org. Since bundle install gems from rubygems.org, you get the erroneous code which will not work with ruby 1.9.3. You can bypass the default by passing the git repository as source and get the current code but then it will get into some other trouble. I decided to better let it go  🙂

Opensource Tools

Surfing for free load testing tools , I come across below options. jmeter look best to me among them. jmeter have excellent documentation

->Pylot

-> Apache Jmeter

-> Tsung

-> Siege

Paid Load Testing Tools

I search for some paid tool also. Being paid, they are far better then other available options, at least I felt so 🙂

-> Neotys

They are providing free trial for one month. I installed it to give a try and find it amazing. It is easy to use with proper documentation and video tutorial . They also provide 24×7 technical support to help you out of any problem. Since, they have detailed explanation of how to, I managed it myself without any problem. I suits all my needs.

-> Load runner 


1 Comment

multibox multiserver deployment with mina

I have explained how to deploy your rails app using mina in this post. Here, I will explain how to do deployment on multiple servers say demo, staging, testing, production etc. The logic is simple, we need to laod the configuration dynamically depending on the server we want to deploy. two things provided by mina is helpful. first it allow you to define your own task with task keyword and invoke other methods with invoke keyword. we will make use of both of them. we will separate all setings from deploy.rb file to a separate file, say deploy_conf.yml. The steps are almost same as I have explained here for single deployment. So go to the last post for step by step explaination

STEP 1: create a file deploy_conf.yml in config folder

This file will contain all the configuration settings for different server. Say, my demo and staging server content in it is as below

staging:
    domain: trantor@192.168.173.52
    deploy_to: /var/www/my_staging_app.com
    repository: git@github.com:skycorporation/my_app.git
    branch: master
    rails_env: staging

demo:
    domain: trantor@192.168.173.52
    deploy_to: /var/www/my_demo_app.com
    repository: git@github.com:skycorporation/my_app.git
    branch: master
    rails_env: demo

STEP 2: Setting the Staging server.

STEP 3: Create empty directory on server manually which required by mina as prequisite.  you should repeat this on each server

STEP 4: install and init mina

STEP 5: write the deployment code to deploy.rb file. Iam showing here only the lines which is added to code from this post

….

require ‘mina/rvm’ #from last post…add below code after it

require “yaml” #need to parse the yml file containing our configuration

YAML.load(File.open(‘config/deploy_conf.yml’)).keys.each do |server|
#we will loop over each server in deploy_conf.yml file and dynamically generate setup and deploy task for that
  desc %{Set up #{server} for deployment}
  task “setup_#{server}” => :environment do # for say demo server, this line is like task setup_demo
    load_config(server) # this will call load_config method below which will the load the configuration for that server
    invoke :setup # will invoke the setup method
  end

  desc %{deploy to #{server} server}
  task “deploy_to_#{server}” => :environment do # for say demo server, this line is like task deploy_to_demo
    load_config(server) # this will call load_config method below which will the load the configuration for that server
    invoke :deploy # will invoke the deploy method
  end
end

def load_config( server)
  thirdpillar_config =YAML.load(File.open(‘config/deploy_conf.yml’)) #this will load the yml file and return a hash for each server
  puts “———-> configuring #{server} server”
  set :domain, thirdpillar_config[server][‘domain’] #retriving the domain value for current server
  set :deploy_to, thirdpillar_config[server][‘deploy_to’] #retriving the deploy_to value for current server
  set :repository, thirdpillar_config[server][‘repository’]
  set :branch, thirdpillar_config[server][‘branch’]
  set :rails_env, thirdpillar_config[server][‘rails_env’]
end

set :shared_paths, [‘config/database.yml’]  #from last post, copy remaining code from there below

……

STEP 6: run mina setup_staging on the terminal

$mina setup_staging #this will call your custom method for setting up server

STEP  7: run mina deploy_to_staging on the terminal

$mina deploy_to_staging #this will call your custom method for deploying to staging server

Similarly, running mina deploy_to_demo will deploy to your demo server.


5 Comments

rails deployment with mina

Recently, I need to automate server deployment on staging machine. I have used capistrano earlier, but decided to give a try to new deployment gem for rails : mina . You can find very good documentation here , which make the things very easy to understand and implement. Before, trying the code on staging server directly, I tried it on my friend suresh machine, as my target server.

STEP 1: Setting the Staging server

Staging Machine (I regarded my friend machine as Staging server)
IP: 192.168.173.52
username: trantor
password: xyz4154
system name : trantor-MS-7788 which basically represent the ip 192.168.173.52

You should able to ssh trantor@192.168.173.52 from your terminal.   see how to ssh to remote machine here

STEP 2: Create empty directory on server manually which required by mina as prequisite
-> /var/www/my_app.com #this folder will become your deploy_to path
-> /var/www/my_app.com/shared/config/database.yml # mina will take database file from here
-> install rvm if you want to use it
-> create the database to be used

Note : Your task on staging server is over with step 2. remaining step is to be done in your own machine except step 9

STEP 3: adding staging.rb file in environment folder
this will contain configuration specific to staging server. add your required configuration to it
STEP 4: Add mina to your gem file and run bundle install
gem ‘mina’ #in gem file
$bundle install # on terminal

STEP 5: Initiating mina
$ mina init
—–> Created ./config/deploy.rb
Edit this file, then run `mina setup` after.

STEP 6: edit the /config/deploy.rb file as instructed there . the final content is as below

require 'mina/bundler'
require 'mina/rails'
require 'mina/git' #use it if your are using git repository
require 'mina/rvm' #use it if you have installed rvm on your server
set :domain, 'trantor@192.168.173.52' # the server to which you ssh
set :deploy_to, '/var/www/my_app.com' #create this folder manually on server, 
                                       mina will deploy to this
set :repository, 'git@github.com:skycorporation/myproject.git' # your repository url, 
                                           this repository must be accessible on server, see here
set :branch, 'master'
set :rails_env, 'staging' #default is production, 
                          so this line is must to deploy on staging

set :shared_paths, ['config/database.yml', 'log'] #will tell mina which things 
                                                 in your project should be taken from shared folder

task :environment do
invoke :'rvm:use[ruby-1.9.3-p194]' # You will create this gemset under setup
end

task :setup => :environment do
  queue! "rvm install ruby-1.9.3-p194" # will install the rvm on server
  queue! %[mkdir -p "#{deploy_to}/shared/log"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]

  queue! %[mkdir -p "#{deploy_to}/shared/config"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]

  queue! %[touch "#{deploy_to}/shared/config/database.yml"]
  queue  %[-----> Be sure to edit 'shared/config/database.yml'.]
end

desc "Deploys the current version to the server."
task :deploy => :environment do
  deploy do
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:db_migrate'
    invoke :'rails:assets_precompile'

    to :launch do
      queue 'touch #{deploy_to}/tmp/restart.txt'
    end
  end
end

STEP  7: run mina setup on the terminal

$mina setup #it will run setup task from deploy.rb in step 6. Your server is now ready for deployment

STEP 8: deploy the code to the server

$mina delpoy

STEP 9: Start the server

cd to installed directory and run rails s -e staging as below

trantor@ trantor-MS-7788$ cd /var/www/thirdpillar.com/current # your current code is in current folder

trantor@ trantor-MS-7788/var/www/thirdpillar.com/current$ rails s -e staging

This will start your application on server with webrick as you use to do on your development machine. To configure it to work with nginx and passenger see this post

STEP 10: See it working

Go to your browser and type http://192.168.173.52:3000 ….press enter….yepieee…the site is UP 🙂

You can find multi box deployment or multi server deployment using mina in this post