Some IT Wisdom Quotes from Twitter

I believe the way we interact with Twitter reflects the mood and the mindset in general we are. Here I collected some of the tweets I've liked and enjoyed reading recently. Let me know if you have others.

The price for free software is your time.

Kelsey Hightower @kelseyhightower

If you don’t end up regretting your early technology decisions, you probably overengineered.

Randy Shoup @randyshoup

Optimize to be Wrong, not Right.

Barry O'Reilly @BarryOReilly

Most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you're probably being slow.

Jeff Bezos, Amazon CEO @JeffBezos

You can't understand the problem up front. The act of writing the software is what gives us insight into it. Embrace not knowing.

Sarah Mei @sarahmei

I love deadlines. I like the whooshing sound they make as they fly by.

Douglas Adams

It is the cloud, it is not heaven.

Everything is a tradeoff... just make them intentionally.

Matt Ranney, Chief Architect Uber @mranney

Microservices simplifies code. It trades code complexity for operational complexity.

Do not strive for reusability, and instead aim for replaceability.

Fred Brooks, @ufried

Signing up for Microservices is signing up for evolutionary architecture. There’s no point where you’re just done.

Josh Evans from Netflix

Inverse bus factor: how many people must be hit by a bus for the project to make progress.

Erich Eichinger @oakinger

If you think good architecture is expensive, try bad architecture.

Brian Foote & Joseph Yoder

API Design is easy ... Good API Design is HARD.

David Carver

If we don’t create the thing that kills Facebook, someone else will.

Facebook’s Little Red Book

The Job of the deployment pipeline is to prove that the release candidate is unreleasable.

Jez Humble @jezhumble

Wait... Isn't forking what #opensource is all about? Nope. The power isn't the fork; it's the merge.

It is not necessary to change. Survival is not mandatory.

W. Edwards Deming

You can sell your time, but you can never buy it back. So the price of everything in life is the amount of time you spend on it.

Hope reading this post was worth the time you spent on it :) Follow me @bibryam for future blog posts on related topics.

Live Photo Streaming from Twitter, Instagram and Tumblr

The Good

UPDATE: this project is dead now. It didn't turn into a successful startup at the end :)
Streaming real time images from Twitter during the London Olympics was fun. Camelympics turned out to be an interesting(mostly for the developers) project:

Claus Ibsen (the core Camel rider) blogged about it and it was mentioned on DZone.

Bruno Borges' talk on "Leverage Enterprise Integration Patterns with Apache Camel and Twitter" at JavaOne conference (slides available here) had the demo inspired by Camelympics.

Streamed millions of images, had more than 1024 visitors on my blog and 5 comments - not bad. All that with less than 128 lines of Camel code, a little javascript to rotate the images on the client side and powered by free AWS micro instance. Camel is tiny enough to do real time streaming, filtering, throttling, serializing, and serving pages with half a gig memory.

The Bad

The Olympics are over. Limiting photos to a certain topic or tag is fine during an events, but not good enough in long term.

Websockets are not widely supported yet. IE doesnt support them, mobile browsers doesn't support them. The only Android browser I found with websocket support is Dolphin Broser, but then 3G networks are not fast enoug to cope with real time (even throttled) image streaming. It needs at least another year before Websockets become widely useful.

Twitter introduced new Developer Rules and limits to make life harder.

As Bruno proved during the live Twitter demo at JavaOne conference, Twitter is the world's largest nsfw photo streaming service, and there is nothing you can do about it (yet).

The Pivot

I extended the application, so in addition to Twitter it also retrieves photos from Instagram and Tumblr in "real time". Tumblr doesn't have a real time api, and Instagram's real time api, based on tags is useless. So I had to come up with some clever ways to give the same real experience for the user.

Added search functionality. It is actually a real time filtering functionality - the application will monitor Twitter, Instagram and Tumblr for photos with a given tag and stream only these images as soon as they are posted. I am not aware of other application doing it.

Decided to replace the custom styling with Bootstrap, it just works.

and the result is...

The experiment continues, looking forward to hear your constructive feedback.

Olympics image loader powered by Camel

This is a very short post about a very simple application. Inspired by the London 2012 Olympics and based on Apache Camel examples, I've created an application that displays twitter images related to the Olympics in real time. It listens for tweets containing images, filters out duplicates, and sends the images to the browser using websockets every 5 seconds. See how simple is the application on my github account.

This is an end-to-end push application: once a user pushes an image to twitter, twitter pushes the image to the camel application, and then the camel application pushes the image to all the clients. It is built using only "free stuff": Twitter's free streaming API, Apache Camel framework, Amazon's free micro instance and done during my free time. Here is the essence:

Line by line explanation

Twitter pushes tweets containing the #Olympics and #London2012 tags to the app:

from("twitter://streaming/filter?type=event&keywords=" + searchTerm)

Log statistical information about the number of messages every minute. Not visible the users:

.to("log:tweetStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")

Extract images from the tweets which has media associated with:

.process(new ImageExtractor())

Put the current number of tweets and images in the message:

.process(new Statistics())

Filter out all the tweets which doesn't contain images:

.filter(body().isInstanceOf(Tweet.class))

Filter out duplicated images, identified by their url:

.idempotentConsumer(header(UNIQUE_IMAGE_URL), MemoryIdempotentRepository.memoryIdempotentRepository(10000))

Log again, the messages which reached this far in the route:

.to("log:imageStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")

Let images go with 5 seconds difference, so the user can enjoy it. Also important - don't block the twitter listener by using callerRunsWhenRejected if the image buffer fills up, Twitter will block you:

.throttle(1).timePeriodMillis(5000).asyncDelayed().callerRunsWhenRejected(false)

Serialize into json:

.marshal().json(JsonLibrary.Jackson)

Push it to the users:

.to("websocket:camelympics?sendToAll=true");

The application can be run also locally and allows filtering images not only for the Olympics but for any keywords passed as argument. Don't forget to use your own twitter oath tokens when using locally though.

Enjoy the Olympics.