2

I've found it's a useful strategy to create relevant queues whenever a queue worker service boots. For instance, on service boot, it checks whether its queues exist already in AWS and creates them if they don't.

Something I'm struggling with now is how to ensure that an SNS subscription for the queue exists. Checking whether the queue exists and creating it if not is easy, subscribing a queue to a particular SNS topic on boot is also easy - but I can't seem to find a sensible way of checking whether a particular queue is already subscribed to a topic!

If it helps, this will be using the Ruby SDK - but I think that the clients across the languages are pretty much all the same. If I can find a sensible way with the cli client it will most likely be available in the ruby sdk.

The only thing I've come up with so far is to list all the subscriptions and then filter them, but this seems a really silly way of doing it as there could be tonnes of subscriptions - I'd rather not have to deal with paging responses on worker boot.

It would be great if there were a way to retrieve whether a subscription is active, based on parameters of the queue ARN and the topic ARN.

Trout
  • 31
  • 3
  • Have you checked how https://docs.ansible.com/ansible/latest/collections/community/aws/sns_topic_module.html does this? It may just have a slightly better way of looping, ie just use a list of all subscription as a dictionary. – jdog Jan 27 '21 at 06:59

1 Answers1

1

In the end I decided to go another way.

I don't think it was actually a good idea to modify things in that way when the worker boots. It can cause problems because if you have to do some emergency modification of the configuration those changes will be reverted the next time your worker instances restart.

Instead, on boot the worker checks if the queue exists. If the queue does not exist, it will create the queue and all its corresponding configuration including SNS subscriptions etc. It deliberately will only create the SNS subscription for the queue if also creating the queue.

This way, if some configuration has to be changed as an emergency in production, those changes won't be "reset" if the worker spins up new instances.

It is a bit annoying, but also safer I think.

Obviously, this is entirely personal opinion and what I went with in the end - but it seems to have served me relatively well so far. If anyone else has any better ideas or disagrees please do let me know.

Trout
  • 31
  • 3