Recently, I have to restructured my code design due to request timeout error on Heroku. We have decided to use Heroku as our production server. Our staging and other instance are running on nginx+apache in our own server. We do have long running process which make third party API calls. The API is slow and take longer time to return the response. We have handled it by setting request timeout to 90 s. So our server wait for 90 second before throwing the request time out error.
Note: a sample code explaining background job is available here
But this piece of code crashed on heroku. Heroku has maximum limit of 30 second for response to return the first byte and it is not configurable, so you can’t increase it further. It is explained here . Heroku is right in its own way. Why they should pay for your poor design. Any request taking more then 10-15 second is crime in itself, in today’s fast pace world.
So the idea, is to process time consuming request in background.
There can we two kind of background job as below:
Detached background jobs:
These are the job which do not need you application to process its output i,e you are just concerned with the result: failed or success.
Some examples are :
1 : sending newsletter or email to users
2: watermarking photos uploaded by users
3: rendering images or pdf
In this case, You push the task to the background. configure retry, to try the failed job set number of time. If you want, you can ignore the failed job or write code to start it again.
Attached Background job:
You application have dependency on output of these job.
Some Examples are:
1: posting some detail outside your application to third party through API
2: Uploading photo or document by the user
Here, You need to keep user on the same page with showing progress bar or rotator and through ajax call keep checking the background worker after, say every 3 seconds about the status of the job, If it is success, you redirect to some other page and if failed, you render the error message to the user.
Choosing between available background job processor gem.
Below is few gems available which handle background job and are very easy to use. The complete list is available here on the rubytoolbox.
=> delayed job
I have listed the gems in order of there popularity. I have used both sidekiq(explianed here) and rescue(explianed here) in my project and found both of them are very easy to use. A comparative analysis of resque and sidekiq is available here , choose whatever suits you.