breaking into the unknown…

background jobs in rails

Leave a comment

Recently, I have to restructured my code design due to request timeout error on Heroku. We have decided to use Heroku as our production server. Our staging and other instance are running on nginx+apache in our own server. We do have long running process which make third party API calls. The API is slow and take longer time to return the response. We have handled it by setting request timeout to 90 s. So our server wait for 90 second before throwing the request time out error.

Note: a sample code explaining background job is available here

But this piece of code crashed on heroku. Heroku has maximum limit of 30 second for response to return the first byte and it is not configurable, so you can’t increase it further. It is explained here . Heroku is right in its own way. Why they should pay for your poor design. Any request taking more then 10-15 second is crime in itself, in today’s fast pace world.

So the idea, is to process time consuming request in background.

There can we two kind of background job as below:

Detached background jobs:

These are the job which do not need you application to process its output i,e you are just concerned with the result: failed or success.

Some examples are :

1 : sending newsletter or email to users

2: watermarking photos uploaded by users

3: rendering images or pdf

In this case, You push the task to the background. configure retry, to try the failed job set number of time. If you want, you can ignore the failed job or write code to start it again.

Attached Background job:

You application have dependency on output of these job.

Some Examples are:

1: posting some detail outside your application to third party through API

2: Uploading photo or document by the user

Here, You need to keep user on the same page with showing progress bar or rotator and through ajax call keep checking the background worker after, say every 3 seconds about the status of the job, If it is success, you redirect to some other page and if failed, you render the error message to the user.

Choosing between available background job processor gem.

Below is few gems available which handle background job and are very easy to use. The complete list is available here on the rubytoolbox.

=> delayed job

=> sidekiq

=> resque

I have listed the gems in order of there popularity. I have used both sidekiq(explianed here) and rescue(explianed here) in my project and found both of them are very easy to use. A comparative analysis of resque and sidekiq is available here , choose whatever suits you.


Author: arunyadav4u

over 7 years experience in web development with Ruby on Rails.Involved in all stage of development lifecycle : requirement gathering, planing, coding, deployment & Knowledge transfer. I can adept to any situation, mixup very easily with people & can be a great friend.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s