First Past The Post

Just by way of a change, I sit here right now in Cambridge, pulling my hair out, which is handy as at least I have hair to pull out, unlike my brother.

Now as a server programmer, this is no real no experience as angry tension seems a natural state when working with distributed systems, but today I have a particularly fun problem.

For a little background, I'm looking into getting some decent horizontal scalability built into a server architecture for a game I'm working on, by making use of Amazon's Simple Notification Service for pubsub type behaviour, in order to allow most requests to be served from in-memory caches, whilst at the same time avoiding session affifinty on my REST front facing servers (which is even more ugly than usual with REST sessions).

For example, when a new member is accepted into a clan, I want to be able to push the updated clan member list to all the front facing servers - this is sensible as the likelihood of that new clan member (or other active clan members) going and checking out the clan profile and other members is ... high ... to say the least, so incurring a SimpleDB roundtrip hit would be sub-optimal.

Therefore I'm looking at pushing cache invalidations or where possibly, updates, out to whatever other nodes are alive via AWS SNS.

 This is a reasonably sensibly put together service, works, but has the annoying (for dev) characteristic that as a push service with somewhat of an affinity to http based notifications, it needs the subscribing server to be accessible from an outside request to do its thing. Fine in production, trickier for a functional test running locally.

Now sometimes I also want to have some durable notification shtuff going on as well - not right now, and not for caching, but later on. I'd been hoping to leave this area off for now, but now it seems I need to dig into this, as I can 'transform' the system from a push to a pull, by using a 'corner' feature of SNS which is the ability to subscribe to an SQS queue, which can then be polled...

... Great! I can do functional tests, I have confidence in everything but the servlet handler for the SNS pushes direct to http, and I've also got the durable SNS+SQS subscriptions implemented as well! ...


Not so much, as whilst the things were working yesterday, it looks like today I can publish as many messages as I like to the SNS topic (and I can even create an email based subscription to veirfy the message is going in), but do I see the messages appear on the SQS queue that is subscribed to the topic? do I heck as like.

So there, grrr

As a first, test, post on this blog, that does pretty well to sum me up - angry programmer

:D