Monday, January 25, 2010

Browser Hacking

Firefox has a cool extension called pyxpcomext that sticks a Python interpreter inside your browser!

But as is it is pretty hard to use unless you are already a browser geek. For example, it is hard to figure out how to access stuff like a page in the browser because all of the Firefox extension documentation is for either C or JavaScript. So you have to sort of read those APIs and translate them thru XPCOM into Python. This is tricky.

So I extended the pyxpcomext example "PyShell" into something a lot more user friendly (it wraps the most common XPCOM stuff into normal Python function calls) and useful. You can access and read details about it here:

Now, what's the point of this, you ask?

Well, what this means is that you can write a Python program that can interact with your browser as you are browsing and even modify the page (sort of like greasemonkey, but in Python) or even "click" the browser to another page for you!

But that's still technology, how about applications?

0. Automation of repetitive browser tasks.

1. Automated test of your live web site if you are a web developer.

2. Sophisticated ad elimination.

3. I'm thinking of a history database that remembers not only the link, i.e. "" but also the contents of a page. Don't you hate it when you browse to an article only to find it gone!

4. Python XML/HTML development. If you are writing a Python back end server then your python is generating HTML. Its pretty cool to just be able to write hprt(" <> My Header < /h1 >") in the Juiced Python Shell, and actually SEE the rendered HTML appear. Its just like the Python standard "print" command but understands html! This really speeds up the dev/test cycle.

But why not...?

...just write an app that grabs pages as an http client? Well, because your app can't execute Javascript so it won't work on all of the new Web 2.0 sites.

...use greasemonkey? Because everyone hates Javascript :-)!! Seriously, because you can spawn multiple threads in your Python program (however, only the main thread can "touch" the browser). Because you have access to the incredibly rich set of Python libraries. Because Python has a real console and debugger. Greasemonkey is a toy to markup web pages. A very cool toy of course! But still your program will always be limited in complexity by inherent Javascript limitations.


  1. Great thing -- and many thanks for the 'domhelper' stuff, what was a great place to start digging the obfuscated Mozilla documentation )

    Doing that, I have made a small improvement to the tab completion feature :

    Currently I substitute the original code for getCompletionsForName() accessing the 'me' thing ( thanks for it as well ) ) -- as the idea of baking the component myself scares as a nightmare -- but if You can point to ( or write ) a short tutorial on cooking these things, I could add the code myself .

    With kind greetings --

  2. Thanks! The tab completion was "pre-fork" work -- it comes from the pyshell example in pyxpcomext. So I'm not familiar with that section. But I will take a look at the code and your patch see if I can integrate them.