Twitter Confirms Realtime Analytics
Twitter Confirms Realtime AnalyticsJul 09 2010 - Technology - Farhad Divecha
Twitter has just confirmed on their engineering blog that they are using the realtime analytics system Cassandra.
For some time now, it had been indicated that Twitter would change their system for storage of tweets from the current MySQL system to Cassandra. However, they have now made it clear that they will continue to store tweets on the MySQL based system, at least for now.
One of the probable reasons they do not want to make this change at present could be the fact that they are trying to sort out the problems they are having with uptime, possibly due to the unexpectedly high traffic volumes they continue to have to deal with.
They will, however, make use of Cassandra for several other purposes. Their geo team has already started using it to store and make queries about places of interest. Their research team is using it to store data mining results for their entire user base. These results can then be used to provide information to @toptweets and local trends.
While it has often been assumed that Twitter would use analytics for monetisation, they have never confirmed this in the past. Now, however, Ryan King of Twitter has said that the analytics operations and infrastructure teams are trying out ways in which Cassandra can be used for large-scale realtime analytics internally as well as externally. He further said that Cassandra will be a part of many new large scale projects from Twitter and they will continue to grow it for a long time to come.