breaking into the unknown…

heroku timeout code H13

Leave a comment

Recently while going live with one of our financial domain project on Heroku we get with this dreaded error.

at=error code=H13 desc=”Connection closed without response”

The error occurring whenever a user try to submit his loan application. Basically a user never able to submit his application due to the above error as the process getting killed due to response taking too long to comeback.

some of the culprit line in logs look like as below:

2015-06-08T19:10:34.297539+00:00 heroku[router]: at=error code=H13 desc=”Connection closed without response” method=POST path=”/loans/credit_request” request_id=0f4a7e2f-28a6-4542-a308-db3a9a72c0e6 fwd=”″ dyno=web.2 connect=0ms service=26007ms status=503 bytes=0
2015-06-08T19:10:34Z app[postgres.32439]: [MAUVE] could not receive data from client: Connection reset by peer
2015-06-08T19:10:34.277236+00:00 app[web.2]: E, [2015-06-08T19:10:34.277078 #3] ERROR — : worker=0 PID:9 timeout (26s > 25s), killing

So basically Heroku is not to blamed. It is right in killing a process if it see no response coming in 25 second.

We know that oue API is the culprit. This is how our whole application work

-> User come to our application build in rails
-> Fill the details related to loan he want to apply and submit it.user see progress bar rotating
-> On submit we capture the data and make a API call to another application written in java.
-> The API do a lot of processing on the data, create citadel certificate, register the user on OpenAm,trigger few emails etc and then send back response.
-> when response is success we reload the page with success message or render the error

It is the second last step in which Heroku seeing a problem. Heroku keep waiting for the response, and when it see that it is exceeding 25 second, it kill the existing process, so the user page never get refreshed and hanged for ever with the progress bar rotating.

So we know the problem…but what will be the solution.

Idle solution is to improve on our API code and make it to return a response in say 15 second(our taking between 25 to 30 seconds). but it is long term plan. we are already live and it is not that we have not worked on reducing the response time(well it is crime if you make user to wait 30 second for response) but since a lot of processing undergo before sending the response, we can’t do much.

So we decided to live with the current response time and fool Heroku to believe that request response cycle is working.

Here is the Plan:
-> push the API call to background job
-> submit the form as ajax
-> In the action which handle submit, trigger the background job and render the job_id , the path to which need to be redirected and other parameter you want to reuse
-> On success of ajax request, trigger another ajax call to a separate method which keep checking the status of the job after a fix interval say 3 seconds.
-> when the second ajax call see that status is complete, it will reload the page

So now, although user see some increase in time due to overhead introduced in checking status after every 3 seconds, Heroku see the request response cycle working as in every 3 second it see a request coming asking for the job status and a response going on with the current status : pending , queued, running etc .

So here is the code:

Add sidekiq to Gemfile and run bundle install

gem 'sidekiq'


  resources :loans
  post 'credit_request' => 'loans#check_credit_request', :as => :credit_request
  get 'check_job_status' => 'loans#check_job_status', :as => :loan_status

write the background code in lib folder say – lib/application.rb

require 'rest_client'
require 'base64'

module LoanPath
  class Application
    include Sidekiq::Worker
    include Sidekiq::Status::Worker
    sidekiq_options :retry => false
    def perform(*args)
      lp_status_code = background_task_output[:lpcode].present? ? background_task_output[:lpcode] : ""
      lp_status_message = background_task_output[:lp_message].present? ? background_task_output[:lp_message] : ""
      lp_data = background_task_output[:lp_data].present? ? background_task_output[:lp_data] : ""
      store :lp_status_code => lp_status_code
      store :lp_status_message => lp_status_message
      store :lp_data => lp_data
      at 100,100, background_task_output[:lp_message] 
                   if background_task_output[:lp_message].present?
  def apply_credit(detail)
      uri = APP_CONSTANTS["credit_request_endpoint"]
      payload = "whatever xml or other data you want to send"
       rest_resource =, {:user => "your username", :password => "xyz", :timeout => 60, :open_timeout =>60})
      credit_submit = payload, :content_type => "application/xml"
      {:lpcode => "LpValid", :lp_message => "Credit Request Submitted Successfully", :lp_data => credit_submit}
    rescue Exception => e
      error_message = "System encountered error, please try later"
      {:lpcode => "LpError", :lp_message => error_message, :lp_data => nil}
  def running_background_job(job_id)
    status = Sidekiq::Status::status(job_id)
    status_message = Sidekiq::Status::message(job_id)
    lp_status_code = Sidekiq::Status::get(job_id, :lp_status_code)
    lp_status_message = Sidekiq::Status::get(job_id, :lp_status_message)
    lp_data = Sidekiq::Status::get(job_id, :lp_data)
    {:status => status.to_s, :status_message => status_message, 
     :lp_status_code => lp_status_code, :lp_status_message => 
     lp_status_message,:lp_data => lp_data}

The loan view to fill the details is as below:

<%= form_tag credit_request_path, :method => :post, :id => "credit-request" do %>
   your form fields
   <%= submit_tag 'Submit Credit Request', :id => 'apply-credit'%>
<% end %>

controller code which handle the loan submit is as below:

def credit_request
  job_id = LoanPath::Application.perform_async("apply_credit", params)
  render :json => {:job_id => job_id , :current_url => loan_url(params[:id])}

def check_job_status
  job_status =[:job_id])
  @application_saved = "Error"
  if (job_status[:lp_status_code].to_s == "LpValid")
    @application_saved = "Success"
    @message = "Credit Application Submitted Successfully."
  elsif (job_status[:lp_status_code].to_s == "LpError")
    @message = job_status[:lp_status_message].present? ? 
               job_status[:lp_status_message] : 
               "System encountered error, please try later"
  render :json => {:status => job_status[:status], :redirect_to => params[:current_path], :message => @message, :application_status => @application_saved}

Now the most important, the  js code which submit the form through ajax and keep checking status is as below

$('#apply-credit').click(function (e) {
    if ($('#accept_tc').is(':checked')) {
        var intervalId = '';
            url: "/loans/credit_request",
            data: $("#credit-request").serialize(),
            dataType: "json",
            type: "POST",
            success: function (job) {
              intervalId = setInterval(function () {
                }, 3000);
            message: null
        return false;
      } else {
        alert('Please read and accept the terms, above, before submitting credit request.');
        return false;

function checkStatus(job, intervalId) {
        url: "/check_job_status",
        data: {
          job_id: job.job_id,
          current_path: job.current_url,
        type: "get",
        dataType: "json",
        success: function (jobStatus) {
          if (jobStatus.status === 'complete' || jobStatus.status === "failed") {
              if (jobStatus.application_status === "Success") {
                  window.location = jobStatus.redirect_to;
              else {

That’s all now Heroku will not complain about Timeout error

Author: arunyadav4u

over 7 years experience in web development with Ruby on Rails.Involved in all stage of development lifecycle : requirement gathering, planing, coding, deployment & Knowledge transfer. I can adept to any situation, mixup very easily with people & can be a great friend.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s