Twitter Streaming API and Apache Wink

During 2009 Twitter launched their Streaming API, which allows developers to get an (almost) real-time notification of tweets. The “filter.json” resource allows some filtering of the content. With CURL you could listen to the Twitter stream like:

curl http://stream.twitter.com/1/statuses/filter.json?track=myKeyWord -umyUserAccount:password

Note that the track parameter could be also send to the stream.twitter.com server (which is a Jetty-powered service) as a POST (-d option).

To have a bit more comfort, one wants to use a (Java) REST API. Apache Wink is pretty nice and listening to the stream is simple:

ClientConfig cc = new ClientConfig();

String logon = "myAccount:myPasswd";
String encodedLogon = new BASE64Encoder().encode(logon.getBytes());

RestClient client = new RestClient(cc);
Resource twitter = client.resource("http://stream.twitter.com/1/statuses/filter.json?track=Berlin");

twitter.header("Authorization", "Basic " + encodedLogon);
twitter.contentType(MediaType.APPLICATION_FORM_URLENCODED_TYPE);
twitter.accept(MediaType.APPLICATION_JSON_TYPE);

// do the HTTP GET
ClientResponse cr = twitter.get();

// we want the input stream
InputStream is =  cr.getEntity(InputStream.class);

try
{
BufferedReader r = new BufferedReader(new InputStreamReader( is, "UTF8"));
while(someCondition)
{
String rawJsonTweetData = r.readLine();
// parse the tweet and submit to your "system"
...

Once you submitted the HTTP_GET you need to transform the returned value to be InputStream. Now the stream is in your hand and you can parse the returned JSON value as you like. From there you can use the streamed data in your system.

I used the stream value to visualize the data with ADF Faces’ comet facility (aka ADS). I created a simple monitoring processor, which receives the tweets and submits them to the ADS subsystem. From there I “pushed” them to the browser, so my ADF Faces table got updated, with the latest tweets.

Note: “Following” a Twitter trend key-word can generate quite a huge amount of data, which could flood your UI (or other processing (sub)system)

There are some libraries available that give similar access to the streaming API. I found the TweetStream Ruby GEM pretty interesting, as you can run it as daemon.

About these ads

Howdy!

Tagged with:
Posted in adf, ajax, apache, java, javascript, jetty, rest
2 comments on “Twitter Streaming API and Apache Wink
  1. [...] The previous blog described the pattern how to connect to the Twitter streaming, by using Apache Wink (a REST client API). [...]

  2. [...] Streaming API Posted March 5, 2010 Filed under: ajax, apache, comet, fun, jetty, web² | Recently I played with the Apache Wink REST Client to access the Twitter Streaming API. Yesterday [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 34 other followers

%d bloggers like this: