4

I'm wondering if I wanted to implement a web service based on java that does web analytics, what sort of architecture should I use. The actualy processing of the Big Data would be done by Hadoop.

However I am not sure what I would need to do to make it asynchronous, or is Hadoop already asynchronous by nature? Could I do something with JMS? If so, how would that fit into the whole thing? Would it be something along the lines that upon receiving the web service request, I use JMS to send a message to Hadoop to handle a particular data, and then wait for it to come back to me? I am not too familiar with asynchronous java so I'm not sure where to start.

yannis
  • 39,647
Jason Madux
  • 83

2 Answers2

2

" I use JMS to send a message to Hadoop to handle a particular data, and then wait for it to come back to me" -- this is using an asynchronous protocol to implement a synchronous interaction.

Is there any need for your program to wait for the reply? If there is couldn't you have another process handle the replies?

There are several asynchronous patterns you could use depending on your requirements:-

  • "Fire and Forget" simply put the request in a queue and trust the receiving process to apply the changes.
  • "Send and Poll" - loop around put all your requests on the queue, process all the replies.
  • "sender and receiver processes" -- have different processes, programs threads or whatever, one or more to send requests, one or more to handle replies.
James Anderson
  • 18,255
1

You might want to consider using something like the Mule ESB: http://www.mulesoft.com/mule-esb-open-source-esb

It's open source, pretty lightweight, can host web services and supports full asynchronous processing of incoming events.

You could start with just creating a lightwieght Mule web service that fires off requests to Hadoop (perhaps via JMS, but you could also just call Hadoop directly). But you'd have the flexibility to add lots more services and capabilities later.

mikera
  • 20,777