Interview with Thomas Imart, Creator of Tweetinvi

From the largest multi-national corporation to the smallest specialty shop, Twitter has become an integral part of most companies’ communication strategy. While direct messaging will always be in the forefront, the ability to analyze Twitter trends is often necessary. But Twitter doesn’t always make that easy.

Over time many of the older APIs have been deprecated or removed entirely. Newer APIs have taken their place, but they often work very differently than one might expect. So we decided to talk to Thomas Imart, creator of Tweetinvi about his offering.

InfoQ: There are many Twitter clients available, what makes Tweetinvi unique?

Thomas: Tweetinvi was born in 2012 during social media research project in which? I needed to analyze a random set of tweets coming from the Twitter Stream API. Initially, I was looking for 3^rd parties libraries in C# but none of them were sufficient to fulfill the requirements of my research. I consequently decided to develop my own library to retrieve the tweets.

Simplicity and Productivity

Since 2012, Tweetinvi has changed a lot but kept its initial mission of simplifying the life of developers. Today with Tweetinvi version 0.9, I believe the productivity it provides is very high - and higher than any other API. As the main developer, my philosophy is to help the users of the API as much as I can. I assisted in the development of various research projects, and I try to reply to any question asked on the forum as fast as I can.

You can find many examples on the documentation page. Here are some of them:

// Publish simple tweet
var tweet = Tweet.PublishTweet("Hello World!");

// Get my Friends
var user = User.GetLoggedUser();
var friends = user.GetFriends();

// Search the tweets containing tweetinvi
var tweets = Search.SearchTweets("tweetinvi");

As I said Tweetinvi is born from the need to get information from Twitter Stream API and here is how Tweetinvi is able to retrieve the Tweets containing the ‘Tweetinvi’ keyword:

// Get all tweets containing tweetinvi
var filteredStream = Stream.CreateFilteredStream();
filteredStream.AddTrack("tweetinvi");
filteredStream.MatchingTweetReceived += (s, args) => { Console.WriteLine(args.Tweet.Text); };
filteredStream.StartStreamMatchingAllConditions();

Flexibility

Tweetinvi is an API that is developed for the developers. Each version of Tweetinvi has an initial changeset that I have always been open to change based on developers’ requests. Therefore, if you need a feature that has not yet been implemented, please feel free to ask on the forum.

Extensibility

One of the strong force of Tweetinvi is the ability it offers to let the developers implement their own queries. Recently, a user asked me whether it was possible to retrieve all the statuses from a UserTimeline since a specific Tweet Id. Tweetinvi implements most of features developers need, but this specific feature was planned to be implemented in a later version. Therefore, I implemented the solution for him in less than a minute:

var since_id = 410493427596288001;
var query = String.Format("https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name={0}&since_id={1}&count=3200", "USER_SCREEN_NAME", since_id);
var tweetsDTO = TwitterAccessor.ExecuteGETQuery<IEnumerable<ITweetDTO>>(query);
var tweets = Tweet.GenerateTweetsFromDTO(tweetsDTO);

There are many other features in Tweetinvi but I will let you discover them by reading the documentation, or browsing the source code.

InfoQ: Over the last year Twitter has been increasing enforcing its rate limits. How does Tweetinvi offer anything to avoid rate limiting issues?

Thomas: Twitter rate limits are mostly affecting applications with a very important number of users. Most developers will rarely be affected by the rate limits if they perform the requests as intended by the Twitter API team.

Tweetinvi and rate-limits

Tweetinvi has the ability to access all the Twitter rate-limits related with an account. You can check the number of operations that are available to you as well as the time remaining before a rate-limit is reseted.

RateLimit.GetCurrentCredentialsRateLimits().StatusesHomeTimelineLimit;

Tips

Select the right type of token

You can request Twitter’s endpoints with either an application Token, or a User token. Developers should most of the time prefer using a user token, as the rate limit will be applied to his specific account and will not affect the rate limits of your application. There are various ways to request a token from Twitter, and Tweetinvi provides two different token generators.

Choose your method wisely!

When it comes to rate-limiting, it is important to choose the Twitter API’s endpoint wisely and base your decision on the number of objects you need to retrieve. You will sometimes need to decide between retrieving populated objects, or simple ids referencing these objects.

For example, it is possible to retrieve 900.000 user ids within the 15 minutes of the rate-limit scope. In the meantime, it is possible to retrieve up to 18.000 populated user objects with the same rate-limit scope.

Use the Streaming API!

I have often worked on projects with developers who did not considered using the Twitter Streaming API. Some of these projects used a set of credentials that allowed them to cope with the rate-limit enforced by Twitter. This is an error!

If you want to get all the Tweets published on your account, you might want to have a Timer and every 30 seconds get the result of Timeline.GetHomeTimeline(), or you can use the UserStream.

The user stream does not have any rate limitation, and it allows you to access updated information related with your account in live. As a result, if I wanted to get all the Tweets that are getting published on a specific user timeline, I would simply use the following code:

var userStream = Stream.CreateUserStream();
userStream.TweetCreatedByAnyone += (s, args) => { Console.WriteLine(args.Tweet.Text); };
userStream.StartStream();

InfoQ: Recently the popular application MetroTwit was shut down do to token limiting. Do you have any thoughts about token limiting by Twitter or just in general?

Thomas: I am not pleased with the decision of Twitter to restrict the access to its API. It is now impossible for large applications to keep using the API with such restrictions.

On the other hand, these restrictions are only applicable to applications with millions of users.

I have to admit that for most of the applications, the Twitter rate limit is well calibrated. I have worked on multiple twitter projects, and I faced problems with the rate limitations on only one research project that required to analyze high quantity of tweets and users.

If you’ve followed all the tips provided in the previous question, there is a very low risk that you reach the rate-limits.

II will be happy to help any developer solving their rate-limit issue, and you can request help at any time on the forums of Tweetinvi

InfoQ: Do you have any other tips for working with the Twitter APIs?

Thomas: Well the first advice I would give to developers is read the Twitter’s documentation. There are lots of information concerning how the Twitter API behaves and what results should be expected for a specific endpoint. The reason is that the Twitter API does not always send the information that we, as Twitter developers expect. Sometimes some behavior are not described in the documentation and you will need to verify the json resulting from your query in order to understand how the API behaves.

For example all queries will return the number of tweets up to the count parameter of a query. But for the case of SearchTweets the count parameter will be overlooked and you can receive less than the expected number of results whilst other results exist!

There are various cases like this one, consequently keep an eye open to the data and add the Twitter’s documentation to your bookmarks!

InfoQ: As far as APIs go, what changes do you foresee Twitter making in the future? And what changes would you like to see?

Thomas: Twitter is a very interesting source to perform researches. It is a great platform to get opinions and feedbacks from users. I would personally look forward to additional features to analyze the tweets by providing more complex search queries, and an increase of the search rate limits.

I think Twitter will attempt to get more information concerning users and trends. Twitter will probably want to sell such information but I would be interested to find some of these information accessible from the Twitter API.

The documentation of the API is pretty good but it is sometimes out of date or does not provide enough details. I would be happy to have a 100% accurate documentation that would allow Tweetinvi to give the best information to the developers.

Also there are some incoherence in the result sets sent by Twitter. For example a Tweet can have a number of Retweets of 42 and at the same time have the field Retweeted set to false. There are various other examples and I think these problems should be fixed in order that we can provide the more accurate information to the developers. In Tweetinvi, I have taken the decision to provide the information given by Twitter “as-is” hoping that errors coming from the API will be fixed in coming updates.

InfoQ: What is coming next?

Thomas: I am willing to make Tweetinvi accessible to more platforms and consequently I am currently developing the first Twitter API for Portable Class Library (PCL) that will be entirely based on the .NET Portable subset. It implies that you will be able to use the Tweetinvi library for projects targeting Windows, Windows 8, Windows Phone and Mono.

About the Interviewee

Thomas Imart is a software developer who specialized in .NET during his first computer science degree. In his last year, he created with his team a Touch Table prototype. After working at Microsoft, he moved in New York to get a degree in Information Systems where he got passionate with application and system design. With the crowdsourcing research team of Stevens Institute of Technology, he built multiple tools to analyze user interactions on social networks. With the need of collecting, storing and analyzing hundreds of million tweets per month, he started building the famous Tweetinvi library. This library is now widely used in the .NET world, and Thomas (also known as “Linvi”) is providing help and tips on the forum to develop your application. Thomas is now living in London, working on different projects, with various technologies such as SQL, WPF, Knockout, MVC, WebApi…