codedecoder

breaking into the unknown…

sample background job in rails

Leave a comment

I have explained the need of background job and when to use it in this post. We need background job worker to execute our background job. I have explained installation of resque here and sidekiq here as job worker. I will explain pushing your time consuming jobs to background with respect to sidekiq. The flow will remain same for resque also.

NOTE : Though the gem used as background processor is sidekiq, the code to trigger these job and getting its status is managed by  sidekiq-status gem. sidekiq-status gem is basically written over sidekiq gem as its wrapper. You should refer sidekiq-status gem if you failed to understand something here

The feature requirement is related to loan application to a bank. You application provide, interface to user to fill up details like name, bank account number, ssn, address etc. on submit of form it go to create action, where you  make API call to bank, who then try to save the user detail in there DB and return response as success or failed with related message. I am just showing structure(in reality it have hundreds of lines of code with complicated logic of validation and other security safeguard) of my current controller code, without any background job.

Let us see our code without any background worker.

controller code

class LoansController < ApplicationController

  def new
    @loan = Loan.new
  end

  def create
    begin
      @loan = Loan.new(params[:loans])
      LoanPath::Application.new.apply_credit(params[:loan]) # this line 
      of code we will move to background later on
      flash[:notice] = "Credit Request Submitted Successfully"
      redirect_to loans_path
     rescue Exception => e
      flash.now[:error] = e.message.present? ? e.message : "System encountered error,try later"
      render :new
    end
  end

end

We have separated the code related to making API call, to another file lib/loan_path/application.rb. Its content is as below.

 
require 'rest_client'
require 'base64'

module LoanPath
  class Application
    def apply_credit(detail)
      details=detail.symbolize_keys
      uri = "http://mybank.com/esb-sunpower/outbound/SubmitForCreditService"
      payload =<<-eos
      <soap11:Envelope xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
      <soap11:Body>
        <Request>
           <Payload>
               <customer>
                  <id>#{details[:CustomerID]}</id>
                  <firstName>#{details[:first_name]}</firstName>
                  <lastName>#{details[:last_name]}</lastName>
                  <bankName>#{details[:bank_name]}</bankName>
                  <accountNumber>#{details[:account_number]}</accountNumber>
                  <routingNumber>#{details[:routing_number]}</routingNumber>
               </customer>
          </Payload>
        </Request> 
      </soap11:Body>
     </soap11:Envelope>
     eos
     rest_resource = RestClient::Resource.new(uri, {:user => "arun", :password => "secret", 
                                                   :timeout => 90 s, :open_timeout => 90s})
     rest_resource.post payload, :content_type => "application/xml"
     end
  end
end

NOTE : The background job works at class level, so the code need to be executed as background must be taken out into a separate class

Iam lucky here , that my initial design already have, the required code in a separate class Application as you see above.

So, what is happening here. in create action we are passing the details filled by the user to apply_credit method of application class, which then convert it into xml payload, and send it to Third Party through restclient call. Till the time response come, user remain on the new page itself, with a rotating image with message , your request is processing. If everything fine I redirect the user to loan index path and if the third party raise some exception, I catch the error message and render the new action. You can see that, I have set the timeout to 90 second , so you can understand how time consuming process it is. This is just an example of a method you should better run in background. You may have any other method in place of apply_credit method here.

So, now we will better our design and process, the apply_credit task in the background. The flow here will be like this.

=> user fill the detail on the Loan new form

=> Submit the data to create action. he can see rotating image with the message “Your request is Processing”

=> Create action will push the API call i,e apply_credit method to the background

=> In create action , we render create.js.erb file which will contain code, which make ajax call to worker, every 3 second, asking about the status of the task send to it

=> If status is success, we redirect it to loan index path as usual, but if failed, we need to capture the error message. The problem here I faced is that, sidekiq do not return any error message but only the status. The message need to be set by you only. So you need to capture the exception in apply_credit method, instead of controller as we are doing it above. Now, since you have captured the exception yourself, It will never fail in background ..so always return the status to be complete. The workaround here is to, use store method of sidekiq, which allow you to define variable, set and retrive value from it. Things get clear when you will see the code below.

Application.rb file content

 
require 'rest_client'
require 'base64'

module LoanPath
 class Application

    include Sidekiq::Worker
    include Sidekiq::Status::Worker
    sidekiq_options :retry => false # it will not retry the 
                  failed action, default is true

    #what ever code you write within perform action will be executed in
     background, see the use of at and store within perform. at will give
     back message at given % of job completd, I have used 100,100 as I want
     message at completion of process. If you have written 5,100 the message
     will be set at completion of 5% and 100%. store allow you to define new
     variable and retrive it later on, I have define lp_status variable to
     which I stored message return by apply_credit method
    def perform(*args)
      status_message = apply_credit(args.first)# we will return status 
      and message from the apply_credit method
      at 100,100, status_message.last if status_message.last.present?
      lp_status = status_message.first.present? ? status_message.first : ""
      store lpstatus: lp_status
    end

  #see that now we are capturing, exception in apply_crdit method itself 
   and make it tot return a array whose first element is status of the process
   and second element is a success or error message. See the use of these
   returned value in credit_submit_status action of the controller
  def apply_credit(detail)
    begin
      details=detail.symbolize_keys
      uri = "http://mybank.com/esb-sunpower/outbound/SubmitForCreditService"
      payload =<<-eos
      <soap11:Envelope xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
      <soap11:Body>
        <Request>
           <Payload>
               <customer>
                  <id>#{details[:CustomerID]}</id>
                  <firstName>#{details[:first_name]}</firstName>
                  <lastName>#{details[:last_name]}</lastName>
                  <bankName>#{details[:bank_name]}</bankName>
                  <accountNumber>#{details[:account_number]}</accountNumber>
                  <routingNumber>#{details[:routing_number]}</routingNumber>
               </customer>
          </Payload>
        </Request> 
      </soap11:Body>
     </soap11:Envelope>
     eos
    rest_resource = RestClient::Resource.new(uri, {:user => "arun", 
                  :password => "secret", :timeout => 90 s, :open_timeout => 90s})
    rest_resource.post payload, :content_type => "application/xml"
    ["LpValid", "Credit Request Submitted Successfully"]
  rescue Exception => e
    error_message = e.message.present? ? e.message :  
                                  "System encountered error, please try later"
    ["LpError", error_message]
  end
 end
 end
end

Below is the code, with API task pushed to the background

controller code

class LoansController < ApplicationController

 def new
  @loan = session[:loan] || Loan.new # from js file you can't render, 
                       but only redirect, so have stored the user filled 
                       data in session in create action to be used here. If
                       session[:loan] is not present, It will create a new 
                       instance of the Loan
  session[:loan] = nil # we have stored its value in variable, so clear it
end

def create
  @job_id = LoanPath::Application.perform_async(params[:loan]) # this is the 
            line which push the job to background and return you the job Id. 
            we have stored it in @job_id variable.You will need this Id to 
            retrive information about its status
  session[:loan] = params[:loan] # create session of @loan data to render on error
  render "create.js" # if you are submitting form from new with :remote => true 
                     i,e ajax form submit, you will not need this line as rails
                    will load it automatically
end

 def credit_submit_status
    status = Sidekiq::Status::status(params[:job_id])#get the status of the job
    lp_status = Sidekiq::Status::get(params[:job_id], :lpstatus) #get the value 
                                     set in the :lpstatus variable you set in
                                      perform method in application class

    status = status.present? ? status : "NotReturned"
    lp_status = lp_status.present? ? lp_status : "NotSet"
    if status.to_s == "failed"
      flash[:error] = Sidekiq::Status::message(params[:job_id])
    elsif status.to_s == "complete"
      if lp_status == "LpError"
        flash[:error] = Sidekiq::Status::message(params[:job_id])
      else
        flash[:notice] = Sidekiq::Status::message(params[:job_id])
      end
  end
    # return back the json data to ajax call
    render :json => {:status => status.to_s, :lpstatus => lp_status.to_s }
  end

end

create.js.erb file with below content

  var jobId = <%= @job_id %>;
  var path = '/loans/new';

  # In case of failure we will redirect to the new action with
    "server_error=true" set in url, the failedActionUrl generate
    it as below
  var failedActionUrl = function(){
     var error = "server_error=true";
      if(path.indexOf('?') == -1) {
          path = path + "?" + error;
      } else {
          path = path + "&" + error;
      }

      return path;
   }

  # seInterval is jquery function which repeat the code within it.
    We will use it to make ajax call to credit_submit_status every
    3 second and redirect according to the status
      setInterval(function() {
        $.ajax({
          url: '/loans/credit_submit_status',
          data: 'job_id=' + jobId,
          success: function(data){
            if(data.lpstatus == "LpError") {
              var currentPath = failedActionUrl();
              window.location = currentPath;
              return false;
            }
            if(data.status == "complete") {
              window.location = path;
              return false;
            }
            if(data.status == "failed") {
              var currentPath = failedActionUrl();
              window.location = currentPath;
              return false;
            }
          }
        })
      }, 3000);
Advertisements

Author: arunyadav4u

over 7 years experience in web development with Ruby on Rails.Involved in all stage of development lifecycle : requirement gathering, planing, coding, deployment & Knowledge transfer. I can adept to any situation, mixup very easily with people & can be a great friend.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s