codedecoder

breaking into the unknown…


Leave a comment

image cropping with paperclip

We are using paperclip for managing files and images in our project . I assume you all know how to use paperclip with its default behaviour .

Our user model managing user profile picture with paperclip look as below :

class User < ActiveRecord::Base
	has_attached_file :picture, 
                    styles: { medium: "300x300#", thumb: "80x80#" }, 
                    content_type: ['image/jpeg','image/jpg', 'image/png'],
                    default_url: "/images/:style/missing.png", 
                    whiny: false, 
                    use_timestamp: false
end

I have used few of the option available with has_attached_file method of paperclip . The complete list is available here .

So what it do, when ever user select his picture from UI, paperclip creates different size of it, here medium and thumb size. you can add more style as per your need.

Now we need to enhance user experience and let him to crop the picture also, which is common now a days. So know user have to go through two steps :

STEP 1 : Crop the picture

This step just allow user to select part of the picture he want to upload rather then the whole pic. You can let the user do that by providing 4 input fields crop_x, crop_y, crop_w, crop_h which is basically x cordinate, y cordinate setting the origin from where image is to cut , with the specified width and height.

But this is not user friendly. A lot of libraries are available which can serve the above purpose in better way.

I used croppic which worked well for me . I will write on its usage on some other day. Meanwhile you can try it yourself.

Our end goal in this step is to get user selected portion of image i,e crop_x, crop_y, crop_w, crop_h

STEP 3 : Make Paperclip to work on cropped part.

So, now paperclip should create different size of cropped part rather then the whole image. To make it work we first need to understand its default behaviour of paper clip and its is in this file . Below are the important lines of this file(I have taken only the line need to understood from each method)

module Paperclip
  # Handles thumbnailing images that are uploaded.
  class Thumbnail < Processor
    
    attr_accessor :current_geometry, :target_geometry, :format, :whiny, :convert_options,
      :source_file_options, :animated, :auto_orient
    
    def initialize(file, options = {}, attachment = nil)
      super
      geometry             = options[:geometry].to_s
      @crop                = geometry[-1,1] == '#'
      @target_geometry     = options.fetch(:string_geometry_parser, Geometry).parse(geometry)
      @current_geometry    = options.fetch(:file_geometry_parser, Geometry).from_file(@file)
    end
    
    def crop?
      @crop
    end
    
    def transformation_command
      scale, crop = @current_geometry.transformation_to(@target_geometry, crop?)
      trans = []
      trans << "-coalesce" if animated?
      trans << "-auto-orient" if auto_orient
      trans << "-resize" << %["#{scale}"] unless scale.nil? || scale.empty?
      trans << "-crop" << %["#{crop}"] << "+repage" if crop
      trans << '-layers "optimize"' if animated?
      trans
    end
  end
end

So you can see that thumbnail class is wrapped up in paperclip module and inherit from processor class . In the initialize method it set @target_geometry and @current_geometry . The main work is handle by transformation_command method . The first line  basically give back scale of the uploaded image and its cropped part . transformation_to method is defined here .

Now we will override these methods of Thumbnail class to suit our need . Read about method overriding in this post .

We are going to create our own custom processor. Lets create a paperclip_processors folder within lib folder and add custom_processor.rb file to it. It will look as below :

module Paperclip
  class CustomCropper < Thumbnail
    def initialize(file, options = {}, attachment = nil)
      super
      if target.crop_w && target.crop_x
        @current_geometry.width  = target.crop_w
        @current_geometry.height = target.crop_h
      end
    end
    
    def target
      @attachment.instance
    end
    
    def transformation_command
      # call crop processing only if user inputs are there
      if target.crop_w && target.crop_x
        crop_command = [
          "-crop",
          "#{target.crop_w}x" \
            "#{target.crop_h}+" \
            "#{target.crop_x}+" \
            "#{target.crop_y}",
          "+repage"
        ]
        crop_command + super
      else
        super
      end
    end
    
  end
end

Our custom_cropper inherit from thumbnail class and override its initialize and transform method. In these method if user inputs for crop is not given, super is called which exhibit the behaviour of Thumbnail without any modification . This condition is put in place as sometime user sync his data from linkedin and at that time crop inputs is not available or say we have created some user in from seed.rb file which face the same issue.

Now our custom processor is defined, we just need to modify the user model to call our custom processor explicitly and so the default thumbnail processor will be not called . The user model will now look as below :

class User < ActiveRecord::Base
	has_attached_file :picture, 
                    styles: { medium: "300x300#", thumb: "80x80#" }, 
                    content_type: ['image/jpeg','image/jpg', 'image/png'],
                    default_url: "/images/:style/missing.png", 
                    whiny: false, 
                    use_timestamp: false,
                    processors: [:custom_cropper]

        attr_accessor :crop_x, :crop_y, :crop_w, :crop_h
end

We have defined attr_accessor for the cordinates and width height.

The controller code will look something like as below

  def upload_profile_pic
    @user.crop_x = params[:imgX1].to_i
    @user.crop_y = params[:imgY1].to_i
    @user.crop_w = params[:imgW].to_i
    @user.crop_h = params[:imgH].to_i
    @user.picture = params[:user][:picture]
    @user.save
  end

That’ s all you need to make your Paperclip custom processor .

 

Resources :

https://github.com/thoughtbot/paperclip

http://www.rubydoc.info/gems/paperclip/Paperclip/ClassMethods

https://github.com/thoughtbot/paperclip/blob/master/lib/paperclip/thumbnail.rb

http://www.rubydoc.info/gems/paperclip/Paperclip/Processor


Leave a comment

simple file upload in rails

I have been doing file upload in rails using attachment_fu in initial days and then paperclip since its arrival. But have never used simple file upload i,e without using any gems. Recently I have to redesign the paperclip upload functionality of one of my application(will discuss in some other post), So thought of starting from beginning i,e how simple uploader works, what benefits paperclip or other available gem adds to it etc .

I am assuming you already have some rails 4 application running. We will add photo upload feature to it. This will be our flow.

->There will be a upload button on new photo page.
->User upload a pic using the simple uploaded.
-> On successful upload he will be redirected to show page, displaying the uploaded image.

generate the model and controller with below command on your project root from terminal

$ rails g model Photo pic:attachement
invoke active_record
create db/migrate/20150626122526_create_photos.rb
create app/models/photo.rb

If you see the generated migration file, it has below content

class CreatePhotos < ActiveRecord::Migration
  def change
    create_table :photos do |t|
      t.attachment :pic
      t.timestamps null: false
    end
  end
end

Run migration to add the photos table

$rake db:migrate

Once the migration is complete and you chek the schema.rb file, you will see below detail of the photos table.

  create_table "photos", force: :cascade do |t|
    t.string   "pic_file_name"
    t.string   "pic_content_type"
    t.integer  "pic_file_size"
    t.datetime "pic_updated_at"
    t.datetime "created_at",       null: false
    t.datetime "updated_at",       null: false
  end

So you can see that t.attachement in the migration file get converted into below 4 fields.

t.string “pic_file_name”
t.string “pic_content_type”
t.integer “pic_file_size”
t.datetime “pic_updated_at”

They represent internal property of any attachement. When user upload any file, you need to read these values from params and need to update these fields. Let generate the controller now

$ rails g controller photos new show create.

It will generate new, show and create action in photos controller and the corresponding views.
It will also generate get routes for all these actions.

  get 'photos/new'
  get 'photos/create'
  get 'photos/show'

remove them and replace with resourceful routes.

resources :photos

write the file upload code to the photos/new.html.erb

<%= form_for @photo, :url => photos_path, :html => { :multipart => true } do |form| %>
  <%= form.file_field :pic %>
  <%=form.submit 'upload'%>
<% end %>

This simply provide a file upload field with a upload button, which submit data to create action.

The whole controller code look as below:

class PhotosController < ApplicationController
  before_action :set_photo, only: [:show]

  def show
  end

  def new
    @photo = Photo.new
  end

  def create
    @photo = Photo.new
    @photo_object = params[:photo][:pic]
    @photo.pic_file_name = @photo_object.original_filename
    @photo.pic_content_type = @photo_object.content_type
    respond_to do |format|
      if @photo.save
        file_name_filesystem = @photo.id.to_s + @photo_object.original_filename
        photo_path_on_filesystem = Rails.root.join('public','uploads', file_name_filesystem)
        File.open(photo_path_on_filesystem,'wb') do |file| 
          file.write(@photo_object.read)
        end
        format.html { redirect_to @photo, notice: 'Photo was successfully created.' }
        format.json { render :show, status: :created, location: @photo }
      else
        format.html { render :new }
        format.json { render json: @photo.errors, status: :unprocessable_entity }
      end
    end
  end

  private
  def set_photo
    @photo = Photo.find(params[:id])
  end
end

The points to notice here is the create action. If you put debugger(I use byebug) inside the create action and try to see the params, it look as below

(byebug) pp params[:photo][:pic]
#<ActionDispatch::Http::UploadedFile:0xb3b9be60
@content_type=”image/jpeg”,
@headers=
“Content-Disposition: form-data; name=\”photo[pic]\”; filename=\”user1.jpeg\”\r\nContent-Type: image/jpeg\r\n”,
@original_filename=”user1.jpeg”,
@tempfile=#<File:/tmp/RackMultipart20150626-7518-liirs5.jpeg>>

(byebug) pp params[:photo][:pic].content_type
“image/jpeg”

(byebug) pp params[:photo][:pic].original_filename
“user1.jpeg”

(byebug) pp params[:photo][:pic].tempfile
#<File:/tmp/RackMultipart20150626-7518-liirs5.jpeg>

(byebug) pp params[:photo][:pic].tempfile.path
“/tmp/RackMultipart20150626-7518-liirs5.jpeg”

So you can see that params[:photo][:pic] is an object in itself from which we can get the name of the file, its content etc.

We have first saved file_name and content_type in db, and then write the files content in public/uploads directory of our project. So that when user visit show page next time we will show him the uploaded image.

NOTE: Instead of writing the content to filesystem, you can send it to any API which handle its storage or do whatever with the data you want to do.

Now in show.html.erb we have below code to render the uploaded photo.

 
<% pic_path = @photo.id.to_s + @photo.pic_file_name%> 
<%img src="/uploads/<%=pic_path%>" alt="" />

So from above implementation, now we can see what paperclip or other gem add to it:

-> You do not need to handle writing the uplaoded file to filesystem yourself. Gem will take care of it
-> Gem can handle storing the uploaded file on ThirdParty storage like amazone
-> They can provide further processing on image like creating tumbnail, applying affects etc.


Leave a comment

what are the components of rails ?

rails is responsible for popularity of ruby to a great extent. Rails 4.2.1 is out and it summarizes itself with below line, each words are true to itself.

Ruby on Rails is a full-stack web framework optimized for programmer happiness and sustainable productivity. It encourages beautiful code by favoring convention over configuration.

As rails is maturing , it is refactoring and restructuring itself. some of the features are removed from core rails library and build into separate gem, which you can include or exclude as per your needs. For example, observers are part of ActiveRecord before 4.0 but after that it is moved to separate gem rails-observer .

Note that rails is gem in itself . It has structured itself in different modules or component , each providing a specific functionality. When you create a new rails application with command below:

$ rails new demoapp

By default all the components get loaded due to below line of code in config/application.rb file

require File.expand_path('../boot', __FILE__)

require 'rails/all'

Bundler.require(*Rails.groups)

module Demoapp
  class Application < Rails::Application
   ----more code ...
  end
end

You are using require ‘rails/all’ , which ask rails to load all the components. Read here about rails booting and loading. But it is unnecessary to load all the components if you do not need it. say your application do not need to send any email, so it is better to not load ActionMailer component or module. Remember that, heavy your application is , more time it will take to load, so keep unwanted thing out.

Well for this you must know what are the components and what it do for you. Below goes the list :

=> ActionMailer :

It manage email support for rails. Compose, deliver, receive, and test emails using the familiar controller/view pattern. First-class support for multipart email and attachments.

=> ActionPack :

It is responsible for mapping the incoming request i,e URL to a controller action and then rendering the corresponding  view. Internally, it is using rack to communicate with the server. It also manage view caching

=>ActionView :

It is responsible for rendering the view . You may not need it if you expose service as an API rather then a interface .The view rendering is done through external gems like erubis for .erb which is the default(you do not install it but get installed as rails dependency. you will see it if you do gem list on your console) or say haml for .haml templates.

=>ActiveJob :

Active Job is a framework for declaring jobs and making them run on a variety of queueing backends. The basic idea is to provide support for creating job to be run later, which can be then handled with other gem build over it. So to say ActiveJob will provide to infrastructure to create the job, which can be then handled by any of the gem sidekiq or resque or delayed job etc .

=>ActiveModel :

They are responsible for model name introspections, conversions, translations and validations . Basically ActiveModel implementations are ORMs (see ActiveRecord, below), but they can also use non-relational storage like MongoDB, Redis, Memcached or even just local machine memory.

 =>ActiveRecord :

They are responsible to map your model with your ORM database. Remember when you do @user.save, the user detail directly get saved in users table.this mapping is getting handled with ActiveRecord. You do not need it if your model do not interact with database . all the callbacks like before_save etc and associations between model are covered under it.

=>ActiveSupport :

ActiveSupport is a compatibility library including methods that aren’t necessarily specific to Rails. You’ll see ActiveSupport used by non-Rails libraries because it contains such a lot of useful baseline functionality. ActiveSupport includes methods like how Rails changes words from single to plural, or CamelCase to snake_case. It also includes significantly better time and date support than the Ruby standard library.

=>Bundler :

Bundler manages an application’s dependencies through its entire life, across many machines, systematically and repeatably. It is responsible to manage all your gem dependency .

=>Railities :

Railties is responsible for gluing all frameworks together. Overall, it:

->handles the bootstrapping process for a Rails application
->manages the rails command line interface;
->and provides the Rails generators core.

=>sprockets-rails :

 It use to manage your assets pipeline introduced since rails 3.1. Initially it was a core feature of rails but has moved into separate gem sprockets-rails since rails 4.0, so that it can be refactored and develop independently . If you have used it with before rails 4, you have observed the log containing details of all the assets requested which is annoying. but after rails 4.0 when it is moved as separate gem, The configuration option config.assets.logger has been removed. Sprockets will no longer pollute your logs when assets are compiled. Nevertheless, Action Pack will continue to log when assets are requested from your server if they are not precompiled. Also there is increase in performance, now assetes get compiled in about 2 second rather then 8 to 10 second

=> Gems:

They are independent library, which you keep adding as per your need.


2 Comments

how rails boot : order of config files loading

Thinking on it for long, but only today get time to delve into it. Have you ever get curious of what going under the hood when you start the rails server, how your rails application get booted, when your gem gets loaded and config files get executed. Well there goes a lot of thing between you type rails s on your project folder from terminal and go to localhost:3000 in your browser.

I tried to see it myself. To get the basic idea I gone through rails doc on initialization . It is quite elaborative make me lost in the details. So I figure out my own way to get the sense of things : “I use puts statement in all the important files, relevant as per the rails doc”

Below is the output on console when I started the server (I am using rails 4.1.9):

$ rails server
	Iam in Gemfile
	Iam in config/boot.rb
	Iam in config/application.rb
	=> Booting Puma
	=> Rails 4.1.5 application starting in development on http://0.0.0.0:3000
	=> Run `rails server -h` for more startup options
	=> Notice: server is listening on all interfaces (0.0.0.0).
	=> Ctrl-C to shutdown server
	Iam in config/environments/development.rb
	Iam in config/initializer folder
        Iam in config/routes.rb
	Iam in config/environment.rb
	Iam in config.ru
	Puma 2.8.2 starting...
	* Min threads: 0, max threads: 16
	* Environment: development
	* Listening on tcp://0.0.0.0:3000

 

So, now you can see the order in which the files is getting loaded are as below:

Gemfile   -> boot.rb    ->  config/application.rb   -> basic server message get printed  -> config/environments/development.rb  ->  files in initializer foler  ->  config/routes.rb  ->  config/environment.rb  ->  config.ru

Now lets see, how and from where they are getting loaded at the code level .

Step 1: Starting the Server

The story begin when you start the server with the below command

$rails server

any command which run from terminal is a executable, residing in one of the folder in load path of your system. You can see it with below command:

$PATH
-bash:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

all the location separated by : is searched by your system to run any command you type on terminal, So it means rails executable (rails executable get installed when you install a rails gem) must be present in any one of these position.

But wait, what the second argument server is for ?  well it is argument to command rails . You can run without it

$ rails
Iam in Gemfile
Iam in Gemfile
Iam in boot.rb
Usage: rails COMMAND [ARGS]

The most common rails commands are:
 generate    Generate new code (short-cut alias: "g")
 console     Start the Rails console (short-cut alias: "c")
 server      Start the Rails server (short-cut alias: "s")
 dbconsole   Start a console for the database specified in config/database.yml
             (short-cut alias: "db")
 new         Create a new Rails application. "rails new my_app" creates a
             new application called MyApp in "./my_app"

So you can see that rails command still run without the server option, but it give you message how to use it properly.

O.K, lets find what code the rails executable have. It may be present in any of folder within the path above. search each of themfor rails . You will get it in bin. Open with nano command to see its content

$ nano bin/rails

#!/usr/bin/env ruby
begin
  load File.expand_path("../spring", __FILE__)
rescue LoadError
end
APP_PATH = File.expand_path('../../config/application',  __FILE__)
require_relative '../config/boot'
require 'rails/commands'

So the code here first try to load a file spring, which basically have the code to load the gems as below:

$ nano bin/spring

#!/usr/bin/env ruby

unless defined?(Spring)
  require "rubygems"
  require "bundler"

  if match = Bundler.default_lockfile.read.match(/^GEM$.*?^    spring \((.*?)\)$.*?^$/m)
    ENV["GEM_PATH"] = ([Bundler.bundle_path.to_s] + Gem.path).join(File::PATH_SEPARATOR)
    ENV["GEM_HOME"] = ""
    Gem.paths = ENV

    gem "spring", match[1]
    require "spring/binstub"
  end
end

The remaining line of rails executable will craete a constant variable holding path of application.rb, load the boot file and the last line will load the commands file(located in installed rails gem at rails/railties/lib/rails/commands.rb )

So till now, you get to know, how the rails server start, why Gemfile are loaded first, followed by boot.rb file.

So we have understand upto :  Gemfile  -> boot.rb

 

Step 2:  Insight into boot.rb file:

We will look into boot.rb file and see how it influence the remaining order of things. It has below line of code

# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

require 'bundler/setup' if File.exist?(ENV['BUNDLE_GEMFILE'])

So boot.rb file do not have any other influence on the remaining sequence of things. It just install all your gems and done with its role

 

Step 3: Insight into way application.rb is loaded

we have noticed that the last line in rake executable is require ‘rails/commands’ , it basically load the file  rails/railties/lib/rails/commands.rb . It has below content

#https://github.com/rails/rails/blob/master/railties/lib/rails/commands.rb
ARGV << '--help' if ARGV.empty? 
aliases = {   "g"  => "generate",
  "d"  => "destroy",
  "c"  => "console",
  "s"  => "server",
  "db" => "dbconsole",
  "r"  => "runner",
  "t"  => "test",
}

command = ARGV.shift
command = aliases[command] || command

require 'rails/commands/commands_tasks'

Rails::CommandsTasks.new(ARGV).run_command!(command)

So you can see that, the above code creating aliase for different commands. now you know how you can use rails s also in place of rails server .

The last two line is important. It load command_task.rb file at(rails/railties/lib/rails/commands/commands_tasks.rb) and the last line run the command you passed i,e for server command , it becomes

Rails::CommandsTasks.new("server").run_command!(command)

basically the above line execute below line of code from the file rails/railties/lib/rails/commands/commands_tasks.rb

    
 #https://github.com/rails/rails/blob/master/railties/lib/rails/commands/commands_tasks.rb
  def server
      set_application_directory!
      require_command!("server")

      Rails::Server.new.tap do |server|
        # We need to require application after the server sets environment,
        # otherwise the --environment option given to the server won't propagate.
        require APP_PATH
        Dir.chdir(Rails.application.root)
        server.start
      end
    end

from here you can see that APP_PATH which store the application.rb file get loaded and the the server get started.

So upto now, you should have clarity on below sequence of flow

rails server -> Gemfile loaded -> boot.rb loaded -> application.rb loaded

 

Step 4: insight into server.start

in the above code after config/application is loaded(require APP_PATH), directroy changed to rails root folder and  server.start is called. The code is as below:

#https://github.com/rails/rails/blob/master/railties/lib/rails/commands/server.rb
module Rails
 class Server < ::Rack::Server
   def start
    print_boot_information
    trap(:INT) { exit }
    create_tmp_directories
    log_to_stdout if options[:log_stdout]

    super
   ensure
    puts 'Exiting' unless @options && options[:daemonize]
  end

  def print_boot_information
   url = "#{options[:SSLEnable] ? 'https' : 'http'}://#{options[:Host]}:#{options[:Port]}"
   puts "=> Booting #{ActiveSupport::Inflector.demodulize(server)}"
   puts "=> Rails #{Rails.version} application starting in #{Rails.env} on #{url}"
   puts "=> Run `rails server -h` for more startup options"

   puts "=> Ctrl-C to shutdown server" unless options[:daemonize]
  end
 end
end

So you can see that the start method doing basic stuff  like printing the boot message(the message you see after application.rb file loads), making temp directory etc and calling super, which means calling the method of same name from the parent class i,e from the Rack::Server class. Rack::Server is defined in another Gem rack .

 

Step 5: look into ::Rack::Server class

https://github.com/rack/rack/blob/master/lib/rack/server.rb
module Rack
  class Server

    attr_writer :options

    def initialize(options = nil)
      @options = options
      @app = options[:app] if options && options[:app]
    end

    def app
      @app ||= options[:builder] ? build_app_from_string : build_app_and_options_from_config
    end

    def start &blk
      # ...

      server.run wrapped_app, options, &blk
    end

    def server
      @_server ||= Rack::Handler.get(options[:server]) || Rack::Handler.default(options)
    end

    private
      def build_app_and_options_from_config
        if !::File.exist? options[:config]
          abort "configuration #{options[:config]} not found"
        end

        app, options = Rack::Builder.parse_file(self.options[:config], opt_parser)
        self.options.merge! options
        app
      end

      def build_app_from_string
        Rack::Builder.new_from_string(self.options[:builder])
      end

      def wrapped_app
        @wrapped_app ||= build_app app
      end
  end
end

So here you can see start calls wrapped_app function which call build_app function to which app function is passed which in turn call  build_app_and_options_from_config function, which have the code to load all the files in config folder.

Now you can understand, how all the files in config folder in rails are loaded automatically. The files are loaded from config folder in alphabetical order, the folder getting the first priority i,e config/environment folder/*.rb files  ->  config/initializer/*rb files -> other files in config folder

So we are done with the whole loading process below:

rails server -> Gemfile loaded -> boot.rb loaded -> application.rb loaded ->config/environments/development.rb   loaded -> config/initializers/* loaded    ->  config/environmet.rb loaded -> config.ru loaded -> server started

 


3 Comments

security in rails

Any thing on the net is vulnerable to attack. The threat can be at any layer: databaseweb server or web application. Some of the common security threat are : Cross Site Scripting (XSS), Cross Site Request Forgery (CSRF, XSRF), Man in the Middle (MITM)SQL Injection (SQLI)Mass Assignment & Parameter Injection etc . There is nothing like complete protection. There is a proverb in hindi “taala sarifon k liae hota hai…choron k liae nahi” , means “Lock on the door can prevent good peoples from entering the house not the thieves“. But at least the lock can make the thieve to struggle for entry and provide 60% security to our house.

Rails framework, provide 70% inbuilt security if you use all its feature properly and adhere to its security guideline . Web Security is a wide field in itself. I will keep updating this blog as I keep exploring, new security measure . Meanwhile If you want to explore it yourself, you can read the below resource :

=> Ruby on Rails Security Guide
=> Ruby official Security page
=> Rubyon Rails Security mailer list

Defence 1 : Protection against Mass Assignment attack.

Many a time, you have used @user = params[:user] in your code . This is a case of mass assignment i,e any thing coming in params[:user] will be assigned to object @user and when you save the object all of them get saved into database. Now here a attacker come, will modify the form in firefox console and add another input field for user with :admin => true , now when you do @user = params[:user] and save the object , he will get admin privilege and can do any thing now onward.

So simple to break in ..right ??

Not so easy with rails 3 and upper version. Now attributes are prevented from mass assignment by default. So you have to explicitly define the list of attributes which are eligible for mass assignment with attr_protected, or attr_accessible .

attr_accessible :name, :age # it is white listing of attributes for mass assignment

It means only name and age can be mass assigned, if params[:user] cantain any attributes say :admin => true , it will throw error.

attr_protected :admin # it is blacklisting of attributes for mass assignment

It means any attribute other then :admin can be mass assigned.

I suggest you to use attr_accessible approach. I feel it is more safe. say you add a new attribute salary in the user table but forgot to modify your model, it will automatically become eligible for mass assignment with attr_protected approach, but with attr_accessible approach it will throw error. Now if you feel salary safe to be mass assigned, you can add it to attr_accessible list.

 

Defence 2 : protection against cross site scripting (XSS) .

Oh..! You haven’t heard about cross site scripting (XSS) …Read it here . In short, It is execution of script by attacker on your page which may stole cookies session or any other information or break your application.

Let us see a simple example :

let you have a application in which person1 , input a message for person2, which get stored in db,when person2 open his account the message displayed on his page with the code <%= @message %> . But person1 is a bad Guy(remember, mom use to remind us – world is full of bad people) . He entered the message “<script>alert(‘Iam virus, Iam in..I will break your code’)</script>” .

When person2 open his page, what he will see. He will see the message “<script>alert(‘Iam virus, Iam in..I will break your code’)</script>” . No harm occur. Person1 must be cursing Rails. With Rails3 any HTML character in input will automatically escaped, in older version you have to use h() explicitly i,e the input code is <%=h text_field_tag “message” %> . More detail on escaping HTML is available here . So the ill conceived message of person1 get passed to db in harmless form as “&lt;script&gt;alert(‘Iam virus, Iam in..I will break your code’)&lt;/script&gt;” .

Now say, person1 is big crook and able to insert the message “<script>alert(‘Iam virus, Iam in..I will break your code’)</script>” in db. But person2 is again not harmed, as not only input but Rails escape any HTML in the ruby code. But what if you do not want to escape the html tag in the message. You want the message to be displayed as such, because Person3 is a nice person and he send a well decorated message “Arun <strong>I Love You</strong>” , in which I love you is highlighted . You want it to display as “Arun I Love You, rather than “Arun <strong>I Love You</strong>” . So you need to override the Rails default escaping behaviour. below will do that

<%=raw  @message %>

great, person3 message look great, but what is this person1 ill conceived message also get executed, you see the alert box below

xss example

xss example

So, person2 has succeeded in attacking your application . using raw for rendering is risky…will you use that???…I don’t think so . If you still want bolded I Love You, try other alternative than raw : sanitize .

 

 Defence 3 : Protection against CROSS SITE REQUEST FORGERY(CSRF) .

A good example of CSRF is available here . The detail documentation of preventing CSRF attack is available here . So it is more like attack through chat messages, email, images etc. This attack is mostly executed on Authenticated i,e logedIn users. Basically, the cross site scripting attack happen due to your trust on the website, which is not fulfilled while the Cross Site request forgery attack occur due to website trust on you. Website trust you with using it properly, identifying its purpose, content, flow etc. It do not expect you to click on the link like “HOT babes”, “Free tour package”, “your own FB profile link with friend request from a sweet girl” etc. If they are not part of the website on your earlier website, there is great chance that some one attempting CSRF attack.But we are INNOCENT people, we are oblivion of any such attack. Rails come to your rescue.

If you are using rails version 3 or above, you can see below within head tag of layout generated by default.

<%= csrf_meta_tags %>

This line will create a secret token for each user session. If you see the page source, you can see the below line generated by the above code.

<meta content=”sptmtpH+EZYnhrfwPplCQkATltqeHwaBAOJAgyj8CII=” name=”csrf-token” />

The above token value is stored in a hidden filed within each form. It look as below in one of my form for the current session.

<input type=”hidden” value=”sptmtpH+EZYnhrfwPplCQkATltqeHwaBAOJAgyj8CII=” name=”authenticity_token”>

When, my session expire and I log in again, this token will change. attacker can never know this secret as it is generated on fly.So attacker can disguise you to submit a form which look like a form of your web application, but he can’t bypass the authenticity token and rail will throw an error, saving us from the attack.

You can bypass the above default security measure of rails with below line of code, if you find any reason to do so.

 skip_before_filter :verify_authenticity_token

But it is like “Going to war without a shield” .will you do that…??.. NO…I know you are mature enough.

Also, always submit form with post method. Get method is too easy to attack. Post is also not fool proof, but is harder to attack.

match “/launch_all_the_missiles” => “missiles#launch_all”, :method => :get

look crazy…look like any one can destroy the world. post provide better protection against attack. If you want such route in your application write it as post method

match “/launch_all_the_missiles” => “missiles#launch_all”, :method => :post

 

Defence 4 : protection against Packet Sniffing (MITM).

Packet Sniffing is also known as Man IN The Middle attack. It is same as, you written all your secret in a letter , posted it to your lover, but postman read it before delivering. There are many software like FireShip , which  can track the bits crossing the wire, and if the data is in plain text..your game is over. Isn’t good to write the letter in encrypted word before posting, the postman would not have know the secret even if he manage to open the letter. Rails do that encryption for you if you set below configuration in your environment file.

config.force_ssl = true
It will Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies. Now data will be send in encrypted form.

 

 

Defence 5: protection against SQL injection
This is one of the most common and abused attack on any web application, and involve hijacking sql command. let us take a example of a banking application, which allow a loged in user to see all the loan he has applied for. When a user click on any of the listed loan, your application show the details of that loan. Let the user is genuine one, who do not alter the params. let params[loan_id] = “23239”

Being a Rails developer, you may use where clause of Active::Record to retrieve the detail. But, you can construct a where clause in three ways as below.

1.9.3-p194 :021 > @loan = Loan.where(:loan_id => params[:loan_id])
Loan Load (0.1ms)  SELECT `loans`.* FROM `loans` WHERE (loan_id = ‘23239’ )
=> [#<Loan id: 376, user_id: 41, loan_number: “100150”, status: “Declined”, app_name: “Pisa Productions”, app_date: “2014-11-24 08:00:00”, relationship_manager: “”, primary_borrower: nil, owner: nil, guarantor_1: nil, guarantor_2: nil, product: nil, loan_amt: 100000, business_type: nil, finance_rate: nil, payment_amt: nil, years_in_business: nil, terms: nil, created_at: “2014-01-02 11:06:08”, updated_at: “2014-01-02 11:06:08”, sales_force_application_id: “006d000000Gj5n5AAB”, loan_id: “23239”, dealership_name: “Pisa Productions”>]

1.9.3-p194 :020 > @loan = Loan.where(“loan_id = ? ” ,params[:loan_id])
Loan Load (0.4ms)  SELECT `loans`.* FROM `loans` WHERE (loan_id = ‘23239’ )
=> [#<Loan id: 376, user_id: 41, loan_number: “100150”, status: “Declined”, app_name: “Pisa Productions”, app_date: “2014-11-24 08:00:00”, relationship_manager: “”, primary_borrower: nil, owner: nil, guarantor_1: nil, guarantor_2: nil, product: nil, loan_amt: 100000, business_type: nil, finance_rate: nil, payment_amt: nil, years_in_business: nil, terms: nil, created_at: “2014-01-02 11:06:08”, updated_at: “2014-01-02 11:06:08”, sales_force_application_id: “006d000000Gj5n5AAB”, loan_id: “23239”, dealership_name: “Pisa Productions”>]

1.9.3-p194 :022 > @loan = Loan.where(“loan_id = ‘#{params[:loan_id]}’ “)
Loan Load (0.2ms)  SELECT `loans`.* FROM `loans` WHERE (loan_id = ‘23239’ )
=> [#<Loan id: 376, user_id: 41, loan_number: “100150”, status: “Declined”, app_name: “Pisa Productions”, app_date: “2014-11-24 08:00:00”, relationship_manager: “”, primary_borrower: nil, owner: nil, guarantor_1: nil, guarantor_2: nil, product: nil, loan_amt: 100000, business_type: nil, finance_rate: nil, payment_amt: nil, years_in_business: nil, terms: nil, created_at: “2014-01-02 11:06:08”, updated_at: “2014-01-02 11:06:08”, sales_force_application_id: “006d000000Gj5n5AAB”, loan_id: “23239”, dealership_name: “Pisa Productions”>]

So, the sql query generated by Active::Record for the different syntax of where clause is same. But under the hood, the third form of where clause is vulnerable to sql attack but not the first two. Now say the bad guy enter, and he want to know the loan detail of any other person. He will just change the params[:loan id] to make a sql attack. This will do the trick for him.

params[:loan_id] = “‘ OR 1=1–‘”

The above is a sql injection code, you can easily see that, it will always return true as even if all other condition return false 1=1 will definitely going to return true true making the OR to return true at the end.

Again run the above in console. You will get below result.

1.9.3-p194 :024 > @loan = Loan.where(:loan_id => params[:loan_id])
Loan Load (0.3ms)  SELECT `loans`.* FROM `loans` WHERE `loans`.`loan_id` = ‘\’ OR 1=1–\”
=> []
1.9.3-p194 :025 > @loan = Loan.where(“loan_id = ? ” ,params[:loan_id])
Loan Load (0.4ms)  SELECT `loans`.* FROM `loans` WHERE (loan_id = ‘\’ OR 1=1–\” )
=> []
1.9.3-p194 :026 > @loan = Loan.where(“loan_id = ‘#{params[:loan_id]}’ “)
Loan Load (0.3ms)  SELECT `loans`.* FROM `loans` WHERE (loan_id = ” OR 1=1–” )
=>  => [#<Loan id: 376, user_id: 41, loan_number: “100150”, status: “Declined”, app_name: “Pisa Productions”, app_date: “2014-11-24 08:00:00”, relationship_manager: “”, primary_borrower: nil, owner: nil, guarantor_1: nil, guarantor_2: nil, product: nil, loan_amt: 100000, business_type: nil, finance_rate: nil, payment_amt: nil, years_in_business: nil, terms: nil, created_at: “2014-01-02 11:06:08”, updated_at: “2014-01-02 11:06:08”, sales_force_application_id: “006d000000Gj5n5AAB”, loan_id: “23239”, dealership_name: “Pisa Productions”>,
..all othe records]

So you can see that, the first two syntax escape the special character while generating the sql query, thus preventing from sql injection but the last one do not.

Always, be careful while writing your where clause.

Take another example, this is serious one, with your negligence any one can bypass your authentication system. You are stubborn to my advice and goes with your old way of writing the where clause.

User.where(“email = ‘#{params[:email]}’ AND encrypted_password = ‘#{params[:password]}'”)

On your login page the user just need to enter ‘ OR 1=1–‘ in both email and password field, and he will be loged in.

But Rails also not provide full protection against SQL injection, may be they get corrected in upcoming version, but currently, like where, if you not use properly all the following methods are vulnerable to sql injection( READ THE DETAIL HERE) : average ,calculate ,count , maximum ,minimum , sum, delete_all, exists?, find, find_by,join, pluck etc

 

 

REFERENCE :

http://rails-sqli.org/

http://guides.rubyonrails.org/security.html

http://blog.nvisium.com/2014/09/understanding-protectfromforgery.html

 

 


Leave a comment

eager loading in rails

recently I got a interview call from a reputed company and come to know that, I have not so good understanding of data loading in rails. So, I tried to dig deeper into data loading. To start with – data can be loaded in two ways : lazy loading and eager loading. The first one refer to loading associated data when need arises and second one refer to loading the associated data , at the time of retrieving the parent object itself. say you have below association

class Physician < ActiveRecord::Base
  has_many :appointments
  has_many :patients, :through => :appointments
  attr_accessible :name, :specialization, :mobile_no
end
 
class Appointment < ActiveRecord::Base
  belongs_to :physician
  belongs_to :patient
  attr_accessible :reason, :physician_id, :patient_id
end
 
class Patient < ActiveRecord::Base
  has_many :appointments
  has_many :physicians, :through => :appointments
  attr_accessible :name, :mobile_no
end

The corresponding tables for them will look as below :

class CreatePhysicians < ActiveRecord::Migration
  def change
    create_table :physicians do |t|
      t.string :name
      t.string :specialization
      t.integer :mobile_no
      t.timestamps
    end
  end
end

class CreateAppointments < ActiveRecord::Migration
  def change
    create_table :appointments do |t|
      t.string :reason
      t.references :physician
      t.references :patient
      t.timestamps
    end
  end
end

class CreatePatients < ActiveRecord::Migration
  def change
    create_table :patients do |t|
      t.string :name
      t.integer :mobile_no
      t.timestamps
    end
  end
end

Let us create some seed data to experiment with. Add below in your seeds.rb file

Physician.create(:name => "Arun", :specialization => "ENT", :mobile_no => "9569806453")
Physician.create(:name => "Anand", :specialization => "MS", :mobile_no => "9569807853")

Patient.create(:name => "Raman", :mobile_no => "9569807851")
Patient.create(:name => "Kapil", :mobile_no => "9569807852")
Patient.create(:name => "Gita", :mobile_no => "9569807853")
Patient.create(:name => "Santosh", :mobile_no => "9569807854")

Appointment.create(:reason => "Ear pain", :physician_id  => 1, :patient_id => 2)
Appointment.create(:reason => "Teeth pain", :physician_id  => 1, :patient_id => 3)
Appointment.create(:reason => "Teeth pain", :physician_id  => 1, :patient_id => 4)
Appointment.create(:reason => "Surgery", :physician_id  => 2, :patient_id => 4)
Appointment.create(:reason => "Surgery", :physician_id  => 2, :patient_id => 1)

run rake db:seed to populate the above data.

Now we have the association ready along with some dummy data.But before proceeding further, try to understand, why we need eager loading. Say the hospital need to generate a report for the months, indicating the number of patients handled by a individual doctor. You may do something like this.

1.9.3-p194 :052 > @doctors = Physician.all
Physician Load (0.2ms)  SELECT `physicians`.* FROM `physicians`
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :059 > @doctors.each do |doctor|
1.9.3-p194 :060 >     doctor.patients.each do |patient|
1.9.3-p194 :061 >       puts patient.name
1.9.3-p194 :062?>     end
1.9.3-p194 :063?>   end
Patient Load (0.2ms)  SELECT `patients`.* FROM `patients` INNER JOIN `appointments` ON `patients`.`id` = `appointments`.`patient_id` WHERE `appointments`.`physician_id` = 1
Kapil
Gita
Santosh
Patient Load (0.3ms)  SELECT `patients`.* FROM `patients` INNER JOIN `appointments` ON `patients`.`id` = `appointments`.`patient_id` WHERE `appointments`.`physician_id` = 2
Raman
Santosh

So, you can see that for 2 doctors in database, 3 queries hitting the database. the first query was to retrieve all the doctors and 2 queries to retrieve the details of there patients. So it leads to n+ 1 queries hitting database. Imagine the load on database, if there is thousands of doctors. This can be avoided by eager loading

Now we are ready to get into the detail of eager loading.The first thing to remember is that, you can do eager loading in any of the following ways

=> includes
=> preload
=> eager_load

O.K so we have too many ways to do eager loading. Let us see how they differ by trying out them on console .

1.9.3-p194 :039 > @doctors = Physician.includes(:patients)
Physician Load (0.3ms)  SELECT `physicians`.* FROM `physicians`
Appointment Load (0.2ms)  SELECT `appointments`.* FROM `appointments` WHERE `appointments`.`physician_id` IN (1, 2)
Patient Load (0.3ms)  SELECT `patients`.* FROM `patients` WHERE `patients`.`id` IN (2, 3, 4, 1)
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :040 > @doctors.first.patients
=> [#<Patient id: 2, name: “Kapil”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 3, name: “Gita”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 4, name: “Santosh”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :041 > @doctors.first.appointments
=> [#<Appointment id: 1, reason: “Ear pain”, physician_id: 1, patient_id: 2, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 2, reason: “Teeth pain”, physician_id: 1, patient_id: 3, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 3, reason: “Teeth pain”, physician_id: 1, patient_id: 4, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

 

1.9.3-p194 :043 > @doctors = Physician.preload(:patients)
Physician Load (0.1ms)  SELECT `physicians`.* FROM `physicians`
Appointment Load (0.1ms)  SELECT `appointments`.* FROM `appointments` WHERE `appointments`.`physician_id` IN (1, 2)
Patient Load (0.1ms)  SELECT `patients`.* FROM `patients` WHERE `patients`.`id` IN (2, 3, 4, 1)
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :044 > @doctors.first.patients
=> [#<Patient id: 2, name: “Kapil”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 3, name: “Gita”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 4, name: “Santosh”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :045 > @doctors.first.appointments
=> [#<Appointment id: 1, reason: “Ear pain”, physician_id: 1, patient_id: 2, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 2, reason: “Teeth pain”, physician_id: 1, patient_id: 3, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 3, reason: “Teeth pain”, physician_id: 1, patient_id: 4, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

 

1.9.3-p194 :046 > @doctors = Physician.eager_load(:patients)
SQL (0.4ms)  SELECT `physicians`.`id` AS t0_r0, `physicians`.`name` AS t0_r1, `physicians`.`specialization` AS t0_r2, `physicians`.`mobile_no` AS t0_r3, `physicians`.`created_at` AS t0_r4, `physicians`.`updated_at` AS t0_r5, `patients`.`id` AS t1_r0, `patients`.`name` AS t1_r1, `patients`.`mobile_no` AS t1_r2, `patients`.`created_at` AS t1_r3, `patients`.`updated_at` AS t1_r4 FROM `physicians` LEFT OUTER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` LEFT OUTER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id`
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :047 > @doctors.first.patients
=> [#<Patient id: 2, name: “Kapil”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 3, name: “Gita”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 4, name: “Santosh”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :048 > @doctors.first.appointments
Appointment Load (0.3ms)  SELECT `appointments`.* FROM `appointments` WHERE `appointments`.`physician_id` = 1
=> [#<Appointment id: 1, reason: “Ear pain”, physician_id: 1, patient_id: 2, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 2, reason: “Teeth pain”, physician_id: 1, patient_id: 3, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 3, reason: “Teeth pain”, physician_id: 1, patient_id: 4, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

strange, appointment is not loading the through relation appointments and firing a separate query for it .let us try again by including it explicitly.

1.9.3-p194 :064 > @doctors = Physician.eager_load(:patients,:appointments)
SQL (0.4ms)  SELECT `physicians`.`id` AS t0_r0, `physicians`.`name` AS t0_r1, `physicians`.`specialization` AS t0_r2, `physicians`.`mobile_no` AS t0_r3, `physicians`.`created_at` AS t0_r4, `physicians`.`updated_at` AS t0_r5, `patients`.`id` AS t1_r0, `patients`.`name` AS t1_r1, `patients`.`mobile_no` AS t1_r2, `patients`.`created_at` AS t1_r3, `patients`.`updated_at` AS t1_r4, `appointments_physicians`.`id` AS t2_r0, `appointments_physicians`.`reason` AS t2_r1, `appointments_physicians`.`physician_id` AS t2_r2, `appointments_physicians`.`patient_id` AS t2_r3, `appointments_physicians`.`created_at` AS t2_r4, `appointments_physicians`.`updated_at` AS t2_r5 FROM `physicians` LEFT OUTER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` LEFT OUTER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id` LEFT OUTER JOIN `appointments` `appointments_physicians` ON `appointments_physicians`.`physician_id` = `physicians`.`id`
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :065 > @doctors.first.patients
=> [#<Patient id: 2, name: “Kapil”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 3, name: “Gita”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 4, name: “Santosh”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :066 > @doctors.first.appointments
=> [#<Appointment id: 1, reason: “Ear pain”, physician_id: 1, patient_id: 2, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 2, reason: “Teeth pain”, physician_id: 1, patient_id: 3, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Appointment id: 3, reason: “Teeth pain”, physician_id: 1, patient_id: 4, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

you can see that, this time the through association appointment also get eager loaded and no separate query fired for it.

O.K….so let us stop here and analyse the above result.

=> includes and preload made 3 queries to database . eager_load made single query to database
=> the 3 queries made by includes and preload is small queries targeted to the three table physicians, appointments and patients. the single query by eager_load involve join of all the three table
=> the query executed by includes and preload is exactly same, but eager_load query differ from both
=> eager_load do not load the through association implicitly, but you have to specify it explicitly.
=> overall time taken by includes is 0.8ms,preload is 0.3ms and eager_load is o.4ms

So, it is clear that executing a set of small queries through preload is faster than, executing a big join of these queries. Also join suffer from cartesian product overload problem as join produce a number of duplicate record which though not affect the database but hit rails on its back as it have to deal with larger number of small and short lived object.

Here rails show its smartness through includes. Actually internally rails eager load data through preload(i,e set of small queries) or eager_load(i,e through join). include just delegate the eager load to preload or eager_load depending on the nature of query you are making. In simple word, If you are not sure which is more efficient for you : preload or eager_load, just use include and it will smartly decide which one is better and delegate the job to that.

Let us see it in action on console.

1.9.3-p194 :002 > @doctors = Physician.includes(:patients).where(“patients.mobile_no != ?”, “765778989”)
SQL (0.4ms)  SELECT `physicians`.`id` AS t0_r0, `physicians`.`name` AS t0_r1, `physicians`.`specialization` AS t0_r2, `physicians`.`mobile_no` AS t0_r3, `physicians`.`created_at` AS t0_r4, `physicians`.`updated_at` AS t0_r5, `patients`.`id` AS t1_r0, `patients`.`name` AS t1_r1, `patients`.`mobile_no` AS t1_r2, `patients`.`created_at` AS t1_r3, `patients`.`updated_at` AS t1_r4 FROM `physicians` LEFT OUTER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` LEFT OUTER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id` WHERE (patients.mobile_no != ‘765778989’)
=> [#<Physician id: 1, name: “Arun”, specialization: “ENT”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

So, you can see that, this time includes fire a single query through join. Here rails saw that, patient to be retrieved has some condition imposed on it, so delegated the job to eager_load internally rather then preload. Note that you can’t eager load conditional data using preload. try the above query through preload.

1.9.3-p194 :003 > @doctors = Physician.preload(:patients).where(“patients.mobile_no != ?”, “765778989”)
Physician Load (0.3ms)  SELECT `physicians`.* FROM `physicians` WHERE (patients.mobile_no != ‘765778989’)
Mysql2::Error: Unknown column ‘patients.mobile_no’ in ‘where clause’: SELECT `physicians`.* FROM `physicians`  WHERE (patients.mobile_no != ‘765778989’)
ActiveRecord::StatementInvalid: Mysql2::Error: Unknown column ‘patients.mobile_no’ in ‘where clause’: SELECT `physicians`.* FROM `physicians`  WHERE (patients.mobile_no != ‘765778989’)

So we get the error as expected.

Now let us make some conclusion at this point
=> use includes if you want rails to decide, to load data in single join query or small set of queries
=> use eager_load if you want to make single query to db without leaving the decision to includes
=> use preload when you want to load data with set of queries

use of preload do not look convincing…right? . let us see a case, where preload can only fit the need. Say you want the physician whose patient is Raman, but at the same time you want to eager load all the patient of that physician.

The query to find doctor whose patient is raman is as below.
1.9.3-p194 :019 > @doctors=Physician.joins(:patients).where(“patients.name = ?”, “Raman”)
Physician Load (0.1ms) SELECT `physicians`.* FROM `physicians` INNER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` INNER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id` WHERE (patients.name = ‘Raman’)
=> [#]

let us chain includes to the above query to eager load the patients detail

1.9.3-p194 :017 > @doctors=Physician.joins(:patients).where(“patients.name = ?”, “Raman”).includes(:patients)
SQL (0.4ms) SELECT `physicians`.`id` AS t0_r0, `physicians`.`name` AS t0_r1, `physicians`.`specialization` AS t0_r2, `physicians`.`mobile_no` AS t0_r3, `physicians`.`created_at` AS t0_r4, `physicians`.`updated_at` AS t0_r5, `patients`.`id` AS t1_r0, `patients`.`name` AS t1_r1, `patients`.`mobile_no` AS t1_r2, `patients`.`created_at` AS t1_r3, `patients`.`updated_at` AS t1_r4 FROM `physicians` INNER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` INNER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id` WHERE (patients.name = ‘Raman’)
 => [#<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :018 > @doctors.first.patients
=> [#<Patient id: 1, name: “Raman”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

So you can see that here, eager loading returning the patient Raman only as includes , seeing where clause in the query delgate the job to eager_load which create a single join with the given condition, returning only the single patient raman.

Now fire the same query, using preload.

1.9.3-p194 :015 > @doctors=Physician.joins(:patients).where(“patients.name = ?”, “Raman”).preload(:patients)
Physician Load (0.2ms)  SELECT `physicians`.* FROM `physicians` INNER JOIN `appointments` ON `appointments`.`physician_id` = `physicians`.`id` INNER JOIN `patients` ON `patients`.`id` = `appointments`.`patient_id` WHERE (patients.name = ‘Raman’)
Appointment Load (0.1ms)  SELECT `appointments`.* FROM `appointments` WHERE `appointments`.`physician_id` IN (2)
Patient Load (0.1ms)  SELECT `patients`.* FROM `patients` WHERE `patients`.`id` IN (4, 1)
=> [#<Physician id: 2, name: “Anand”, specialization: “MS”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]
1.9.3-p194 :016 > @doctors.first.patients
=> [#<Patient id: 4, name: “Santosh”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>, #<Patient id: 1, name: “Raman”, mobile_no: 2147483647, created_at: “2014-07-21 11:51:18”, updated_at: “2014-07-21 11:51:18”>]

This time you got all the patients of the physician Anand.

So, we are done with our research on data loading in rails.

But wait….Rails still evolving, and it keep changing with each release. The includes has also changed in rails4. Now rails has stopped being super smart. thus it will not automatically delegate the job to eager_load on seing a where clause in the query. instead, you need to pass the reference of the table whose attributes used in the where clause.

Example :

@doctors = Physician.includes(:patients).where(“patients.mobile_no != ?”, “765778989”) # will throw error

@doctors = Physician.includes(:patients).where(“patients.mobile_no != ?”, “765778989”).references(:patients) # will work

 

Reference :
http://guides.rubyonrails.org/active_record_querying.html
http://stackoverflow.com/questions/1208636/rails-include-vs-joins


Leave a comment

rails custom deliver method : sending email through API

I have explained sending email using ActionMailer class of Rails Framework in this post . Go through it first if you not clear on how rail deliver the email. Now our requirement here is that, we do not want the rail to deliver the email for us, But want to get it deliver through ThirdParty email service, which say provide designed templates for the email or advertisement on the email or some other benefits. Basically, we want the email to be delivered through some API rather then Rails. So we need custom email delivery method

Here, you need to understand below two things how action mailer work.

=> ActionMailer create a email object which contain evrything you see in any email : subject, to, from, body, header etc

=> deliver method send the email object to destination using setting specified in the configuration file.

So, our approach will be to keep the first part of action mailer as it is,  and override the deliver method used in the second step.

NOTE : For complete detail on sending email through rails is explained in this post.Here I will mention only the steps which is needed to deliver email through custom method. Step 1 and Step 2 are new addition to the last post and step 3 will override the configuration used earlier. Other things will remain same.

Step 1 : Write Your class which interact with the API

Let us call this file postman.rb and place it in the lib folder of our rails application.

NOTE : define your own code within the deliver! (will throw exception if some error) or deliver (will not throw any exception) method. I have simply shown a sample code from one of my application. It will not work for you as I have used dummy password

module MyFinanace
  class Postman

    def deliver!(message)
      deliver_type = message.header["deliver_type"]
      if deliver_type == "api"
        url = "http://mansel/esb-sunpower/outbound/emailService/"
        username = "mensalTemplate"
        password = "gateway13"
        rest_resource = RestClient::Resource.new(url, username, password)
        rest_resource.post prepare_payload(message), :content_type => "application/xml"
      else
        mail.perform_deliveries = false
      end
    end

     def prepare_payload message
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
        <emailRequest>
          <payload>
            <from>#{message.from.first}</from>
            <to>#{message.to.join(';')}</to>
            <cc>#{message.cc.join(';') unless message.cc.nil?}</cc>
            <bcc>#{message.bcc.join(';') unless message.bcc.nil?}</bcc>
            <replyTo>#{message.reply_to.first.to_s unless message.reply_to.nil?}</replyTo>
            <subject>#{message.subject.to_s unless message.subject.nil?}</subject>
            <body>#{message.body.to_s}</body>
            <contentType>text/html</contentType>
          </payload>
          <messageType>default</messageType>
        </emailRequest>"
    end

  end
end

So you can see that, We have redefined the deliver! method and making API call within it to mansel service. We pass it all the required detail like subject, to, from, body etc in the payload and left up to mansel to deliver the message.

STEP 2: Add your class defined above to the delivery method of rails ActionMailer class

Rails by default support smtp delivery. Since we have added our own class to deliver the email we need to tell ActionMailer about it. We can achieve it with add_delivery_method of ActionMailer . We need to specify it the time our application boot up, so be will create a file say my_mailer.rb in initializer folder of our application and add below lines to it.

require 'postman'
ActionMailer::Base.add_delivery_method :my_finanace_smtp, MyFinanace::Postman

So we have named our custom delivery method as my_finanace_smtp and pointed it to Postman module of MyFinanace class . Now, whenever we start our application, we will have our custom delivery method available to ActionMailer.

STEP 3: configure your environment file to use your custom deliver method

Since now we are using custom deliver method , which will be handled by API outside our rails application, We will replace the configuration used in development.rb file in step 1 of this post with below two line.

  config.action_mailer.default_url_options = {:host => 'localhost:3000'}
  config.action_mailer.delivery_method = :my_finanace_smtp

That’s all, now everything will remain same as in the last post, but your email will be delivered through your custom email deliver method.

Reference:

http://guides.rubyonrails.org/action_mailer_basics.html