November 2015: Job queues

calendar Nov 23, 2015 · 2 min read · messaging pattern queue video  ·
Share on: linkedin copy

Please RSVP at Meetup or Google+.

Food at 6:30 pm. News and Talks at 6:45.

eBook Giveaway: R in Action, Second Edition, Unity in Action and Go in Action

Talks

Messaging patterns and job queues in Go

Gerrit Renker gave a tour of message-oriented middleware (MOM) and job queues, and how Go fits in. He contrasted asynchronous, decoupled messaging ("mail") with synchronous RPC ("telephone"), covering queue and pub/sub endpoint models, brokers, and delivery guarantees (at-most-once, at-least-once, exactly-once). The talk traced enterprise MOM history from the Information Bus through IBM MQ, MSMQ, and JMS to AMQP and RabbitMQ, then surveyed the Go ecosystem: streadway/amqp and rabbit-hole for RabbitMQ, ZeroMQ and nanomsg for high-speed messaging, Shopify's sarama for Kafka, plus NATS, NSQ, Redis (rmq), Gearman, beanstalkd, machinery, and STOMP. It closed with pattern libraries — go-kit (rate limiter, circuit breaker, load balancer, tracing) and go-resiliency (circuit breaker, deadline/timeout, batching, retrier) — and the observation that while MOM breadth is huge, Go's strengths lie in the depth of its concurrency.

Key points:

  • MOM decouples senders and receivers; RPC couples them.
  • Endpoint models: point-to-point vs publish/subscribe.
  • QoS: at-most-once, at-least-once (needs idempotence), exactly-once.
  • Enterprise lineage: TIB → IBM MQ/MSMQ → JMS → AMQP/RabbitMQ.
  • Go clients and platforms: streadway/amqp, rabbit-hole, sarama (Kafka), NATS, NSQ.
  • ZeroMQ/nanomsg trade reliability for performance; job queues include Redis rmq, resque/goworker, Gearman, beanstalkd, machinery.
  • Resilience patterns live in go-kit and go-resiliency.

Gerrit Renker (pdf slides)

The Legacy of Go

Dave Cheney (video)

News

Events