Posts tagged queueing

Apr24

Beanstalk, a Simple and Fast Queueing Backend

queueing beanstalk minion stalker | comments

Web apps are increasingly focused on background jobs. In fact, the term “background job” almost seems inaccurate - the heavy lifting done by worker processes is often the meat of the app’s purpose. The web portion of the app, by comparison, does only the relatively lightweight work of putting job requests into queues, and later presenting the results of jobs as HTML or JSON.

Continue reading »

Sep28

Background Jobs with RabbitMQ and Minion

queueing rabbitmq minion | comments

So: you’re now hip to doing your heavy lifting in the background, via a queueing system like Delayed Job. If not, read The Rails Way's guide to Delayed Job, watch the Railscast about Delayed Job, or read my post on building a queue-backed feed reader.

Continue reading »

Aug13

Threads Suck

concurrency erlang eventmachine queueing | comments

Ricky Ho gives an excellent description of two competing models for concurrency: theads vs. sequential message passing.

Continue reading »

Jun29

Big Trends

databases cloud queueing | comments

The following are what I believe to be the three most important areas of radical (vs. evolutionary) innovation in web application server components over the coming 6 - 18 months:

Continue reading »

Jun24

Backlogs and Request Time

dynos queueing scaling concurrency | comments

When we say that an http request takes 300ms on the server side, that number actually encompasses two phases. One phase is the time the request spends in the backlog, waiting for an available backend process (mongrel/thin/dyno). The second phase is how long it takes to process the request once it reaches the backend process. Conflating these two is a mistake, as it masks important information about what steps you can take to make your app able to handle more traffic.

Continue reading »

May18

Delayed::Job with Sinatra

sinatra queueing | comments

DJ is a great Rails plugin for running a worker queue through your database. But it only works with Rails… or does it?

Continue reading »

Apr15

Building a Queue-Backed Feed Reader, Part 2

queueing ruby rails | comments

In Part 1, we built QFeedReader, a simple and scalable web-based RSS reader backed by DJ.

Continue reading »

Apr14

Building a Queue-Backed Feed Reader, Part 1

queueing ruby rails | comments

Queueing is a critical tool for building truly scalable web apps. Don’t do any heavy lifting in the web processes (mongrels); instead, put jobs on a queue, let background workers do the work, and then display the results to the user in another request.

Continue reading »