Using tweepy on AppEngine Python

This blog post is all about using tweepy on Google AppEngine.

What is Tweepy?
Tweepy is a library written in Pure Python (yeah Pure Python), by JoshTheCoder. There are many libraries in Python for twitter, however my choice for tweepy is because of following reasons

  1. Hyper Active Development
  2. Up to date with twitter APIs
  3. Python 3 supported
  4. OAuth and Cache system
  5. Really simple to understand (my favourite)

How does Tweepy Work?
Tweepy uses basic as well as OAuth for authentication with twitter, and has well defined classes for almost everything that you might need.

What is OAuth?
OAuth is an open protocol, initiated by Blaine Cook and Chris Messina, to allow secure API authorization in a standard method for desktop, mobile and web applications.

For consumer developers, OAuth is a method to publish and interact with protected data. For service provider developers, OAuth gives users access to their data while protecting their account credentials. In other words, OAuth allows a user to grant access to their information on one site (the Service Provider), to another site (called Consumer), without sharing all of their identity.

More details on OAuth can be found at http://oauth.net

Getting started with Tweepy on GAE
Before reading this section make sure you have access to the code sample at http://code.google.com/p/codecontrol-samples/. That will be helpful.

The basic authentication process follows the following steps

  1. Create an OAuthHandler object using the Consumer Key and Consumer Secret. This is something that is provided by twitter when you start with an application on twitter. For more details on Consumer Key and Consumer Secret, visit http://twitter.com/oauth_clients
  2. Next, we have to generate something called as Authorization URL. The authorization URL is a URL which looks like http://twitter.com/oauth/authorize?oauth_token=slyFKiyIiQ1RRHnhIg8AilcYIlgoL37u1goWZILss. Don’t worry about the oauth_token and how to generate this URL. This is something that Tweepy does for you. All you need to do is to call the get_authorization_url method of the OAuthHandler object that we created in step 1.
  3. Alongwith Authorization URL are generated request tokens. These are the keys which are used while asking access from twitter. Make sure we store them in datastore as we will be needing them later.
  4. Your users need to be redirected to the authorization URL page created in Step 2. This is where Twitter will ask permission from it’s user to share their details with our application. If they allow the action. Twitter redirects them to the CALLBACK_URL. This callback_url is a URL on your domain, where users will land after a success. This URL will contain some parameters like oauth_token and oauth_verifier.
  5. Next, we need to lookup for the request token that we saved earlier. Using this request token alongwith consumer key and consumer secret, we create something called as ACCESS_TOKEN. This is the actual token that will allow us access this user’s information on twitter. Again, we must store all these tokens in the datastore so that everytime our user need not authorize us.
  6. Now its all done. We have in our datastore the request token, the access token and the oauth verifier ad oauth token. We will always create an object of OAuthHandler using these data whenever we want to access someone’s twitter information.
  7. To use the tweepy’s twitter APIs, we need to create an instance of tweepy’s API class using the OAuthHandler object created in last step.

Well, that explains pretty much of how tweepy works. If you have found any issues with the code at http://code.google.com/p/codecontrol-samples/, create an issue and i will be more than happy to fix it up.
The code can be directly downloaded from http://codecontrol-samples.googlecode.com/files/gae-tweepy.tar.gz.

A better feed reader

Most of the feed readers – web based, desktop based are more or less similar in their approach. You subscribe to a feed, and they will display all new (or non read) items. It is good. Now this is what a feed reader should do. Some feed readers (like Google reader) go beyond this and allow you to share your favorite feeds with your friends.What more does anyone needs?

Well, there are many more ways in which a feed reader can be enhanced. I wil be talking about the one that i find most useful.

Everything is not important

I am a feed junkie. I go to any website, and if i find anything interesting, i immediately subscribe to its feed. Months later, i realise that not all posts from that website is what i actually am interested in. Let’s take a scenario.Most people are subscribed to Slashdot or LifeHacker or Reddit. I am particularly interested in Programming and Technology. And specifically in Python, C++, WebApps. But what i am getting in my feeds is everything from FORTRAN to JAVA and from Microprocessors to launch of new space craft. This is useless information for me, and what i will be doing probably is – select all, mark as read. If i dont get a chance to read my feeds for a day or two, i will easilt have 1000+ posts and when bulk mark as read, i am probably going to miss out things that might be important for me.

Overcoming the situation

In my opinion, there is only one way to overcome the situation. Get the specific posts from the feeds, in which the reader is interested. So, if i am interested in Python and C++, i will be able to see and read only those feeds that have relevent information about Python and C++.
Again, there are two approaches to make this possible.

Approach 1) Let the reader speak for himself.

When a user is subscribing a new feed say “Programming”, suggest him – This is a very broad category. Would you like something in particular, say C++ or Python? If he says yes, go ahead and keep this thing in memory.

Aproach 2) Find out yourself.

In this approach, the user is initially presented with all the posts corresponding to a feed. A track record is kept for the feeds that the user has actually read and for those feeds for which he has just “marked as read”. Based on these data over a span of time, the system will be generating a set of keywords, which describe a range of interests for one particular user.

One or both of the above approaches can be used. What matters is that the set of keywords for the range of user’s interests in posts should be accurate. This is never guaranteed to be comprehensive, but should be somewhat near to it.

Now things are quite simple. We have the feed, we have the list of posts, and we have the user’s interest. For every post in the feed, scan it. See if it contains things that might interest user. Show him only those posts, that he is interested in. If the system is uncertain about a post, that can be under the “posts you might be interested in” section of the feed reader.

If implemented in this manner, i believe that feed readers can be more popular and good than the present ones.

If any engineer who is working on some feed reader is reading this post, and might consider it for their next release, i would be happy if they could contact me and say “Thanks” 🙂

Make this world a better place.

Cracking Codechef hard problems – Python

<!–
google_ad_client = “pub-4601002425609095”;
google_ad_host = “pub-1599271086004685”;
/* 468×15, created 3/29/09 */
google_ad_slot = “9096275585”;
google_ad_width = 468;
google_ad_height = 15;
//–>

<script
src=”http://pagead2.googlesyndication.com/pagead/show_ads.js&#8221; type=”text/javascript”>
In this blog post, i will be discussing about the hard problems at Codechef.com and how to handle them effectively using Python. This is going to be a “learn by example” post.

So, let’s begin.

Consider the Orders Problem. The problem is big, and i am not going to post it anywhere here.

Now, at one go, you might find it easy, but as you start off with pen and paper, things go a little complicated. A close look will reveal that this is one of the most common sorting technique. Yes, it is Insertion Sorting, that is being done by Sgt Johnny. Let’s analyze how.

The pseudocode for insertion sort goes as follows:


insertionSort(array A)
begin
for i := 1 to length[A]-1 do
begin
value := A[i];
j := i-1;
while j ≥ 0 and A[j] > value do
begin
A[j + 1] := A[j];
j := j-1;
end;
A[j+1] := value;
end;
end;

What we have to do is the opposite of it. We have the sorted list, we have partially the steps covered by each soldier, now we need to find the unsorted list.

Proceeding with the sample input/output given at Orders Problem, we do the following


Input - 0 [1] 0
position - 1st 2nd 3rd

Now move the 2nd soldier 1 place to the left. This is our first iteration. So, at the end of first iteration, we have the following


Position - 2nd 1st 3rd
Modified Input - 1 0 [0]

Next, we will see if there are any further iterations possible. So, we move onto next position in modified input and find that it is 0. This means that the 3rd soldier will not move from it’s place. Now, since there are no more iterations that can be done, we arrive at the position being

2 1 3

which is the solution to the problem.

See if it works good for second sample too. The input is

0 1 2 0 1

So, placing the list in a nice format it becomes. This is iteration 0 (no iteration at all)


Input - 0 1 2 0 1
Soldier - 1st 2nd 3rd 4th 5th

Scan the input from left. See, if there is any non-zero value. Yes, we get a non zero value (1). So, now move the 2nd soldier 1 position left. This is our iteration 1. Note that the first iteration was performed at 2nd element of the Input array. This is important, so that we can start our 2nd iteration at the third element.


At the end of 1st iteration::
MInput - 1 0 2 0 1
Soldier - 2nd 1st 3rd 4th 5th

The third element in MInput (Modified Input) is 2. So, we move the 3rd soldier to two steps on its left. This is what we get at the end of second iteration


MInput - 2 1 0 0 1
Soldier - 3rd 2nd 1st 4th 5th

Is there any scope for further iteration? yes, there is ! So, at the end of third iteration, we get something like


Minput - 2 1 0 1 0
Soldier - 3rd 2nd 1st 5th 4th

Now there is no scope for iterations, and hence we stop and end up with the original list as


3 2 1 5 4

Creating the code
When it comes to programming languages, i am comfortable with only one language – Python. So, i will be posting the code in Python only. Hard luck Java guys 🙂


def SgtJohnnySort(il):
"""Follows the reverse Sgt Johnny Sort and displays the original
unsorted list.
il = input list ex: 0 1 2 0 1
sl = the sorted list; which gradually becomes the unsorted list
"""
n = len(il)
sl = range(1, n+1)
last_moved = 0
for i in range(last_moved, n):
if il[i] != 0:
el = sl[i]
sl.remove(el)
sl.insert(i - il[i], el)
el = il[i]
il.remove(el)
il.insert(i-1, el)
last_moved = i
return sl

In my opinion, this should work. However, i tried this on Codechef, and i could get some runtime error which i am unable to understand. While i am trying to figure out the runtime error, why don’t you have a look at above code and notify if something is wrong 🙂

Getting Started with AppEngine – Pune GTUG

The Pune Google Technology User’s Group (Join Pune-GTUG) will be holding a session on Getting Started with Google AppEngine. The details are as under :

  • Date :: May 16th, 2009
  • Time :: 1600 hrs – 1730 hrs (India Time Zone)
  • Venue :: Dnyanvatsal Commercial Complex
    Survey No. 23, Plot No. 189,
    Karve Nagar,
    Pune, India 411052 (Map)

Speaker :: Pranav Prakash (me 😉

Agenda Topics ::

  1. Little talk about Cloud Computing
  2. Getting Started with Google App Engine Python
  3. Live Examples on Google App Engine
  4. Question and Answer
  5. Introduction to Google App Engine Java (if time permits)

If you are in Pune, and interested to learn about developing scalable web apps in Google Appengine, then you should attend this talk.

See you there.

OpenSource alternate to Dreamweaver

Of lately, i was thinking of switching to open source alternative for DreamWeaver, and i found out Aptana. Aptana is an eclipse based IDE that provides a lot of benefits which but obviously includes being the powers of opensource and freeware. Aptana comes in two flavors – a stand alone IDE and an Eclipse Plugin. You can install either of the two depending on your needs. Personally i dont like Ecipse, so i decided to go for stand alone IDE. You might want to check out the download page for Aptana

After installing Aptana, you need to install CFEclipse plugin for providing ColdFusion support in Aptana.

Installing CFEclipse in Aptana

  1. Download the latest version from the archived software update site: http://www.cfeclipse.org/update/
  2. Unzip the archive file. You should get a folder called “org.cfeclipse.cfml.update.release”.
  3. In Aptana, go through the usual procedure to add an update site (Help -> Software Updates -> Find and Install -> Search for new features to install).
  4. Click on New Local Site…
  5. Choose the folder that you extracted from the zip file (e.g. the location of “org.cfeclipse.cfml.update.release”).
  6. Click Select.
  7. Change the name to something more reasonable such as CFEclipse Local Install.
  8. Click OK and then click Finish.
  9. Follow the installation procedure and prompts… (I am sure you can handle it from now on).

Pretty Simple. That’s it. And now we have an open source alternative for DreamWeaver that is much better than the original one.