Pyjamas and Web2py

UPDATE: Pyjamas has since been renamed Pyjs and is under new leadership. Everything is still backwards compatible.

At this point, if you’ve been following along the posts, you should know how to create a simple web2py application. In this post I’m going to describe how to write a page that can connect to a backend written in web2py. This is another step on the path of having an app on GAE, in fact, the code we write here will get deployed on GAE. This code will also run on any system, including your own computer, and avoids the lock-in some people experience when developing for GAE.

There are 2 ways that I use for communicating with a web server are RESTful JSON calls, and JSON-RPC calls. Instead of covering both I plan on just showing how to use JSON-RPC. I suggest when you design your app first search online for comparisons between REST & JSON-RPC to see the trade-offs.

Before we get into the code, I also have a slight curve-ball. I originally wrote this code using Pyjs, but I’ve switched to using Rapydscript for all my development. In a future post I’ll show how to connect Rapydscript to GAE, which I plan to link here. I suggest reading Alex’s earlier post https://blogs-pyjeon.rhcloud.com/?p=301 for a good summary of various Python to JS compilers and their trade-offs.

web2py Services

Time to code!

First you need to setup your web2py app to use web2py’s services module. This lets your application work as a web service so it can respond to calls from clients, including support for JSON-RPC calls. Inside of models/db.py add the following lines:

from gluon.tools import Service
service = Service()

In the controller, add a call function that returns the service. Then all you have to do is add a decorator to each functions you want to act as services receiving jsonrpc calls and web2py handles the rest. In controllers/default.py I have added the following:

@service.jsonrpc
def myfun(data_from_JSON):
    return data_from_JSON.upper()

def call():
    return service()

The example function here, myfun, will make everything uppercase. Also worth nothing is data_from_JSON is already decoded data from the JSON request.

To access this service use the URI ‘/cyborg/default/call/jsonrpc’. For more information on services check out http://web2py.com/books/default/chapter/29/10#Remote-procedure-calls.

Pyjs Clients

I’ve had this code floating around my computer for a couple years now. I’ve made several minor changes, some just because I wanted slightly lighter code, and some because of changes in Pyjs, but it was originally based on code by Amund Tveit on his blog (http://amundblog.blogspot.com/2008/12/ajax-with-python-combining-pyjs-and.html). This is a simple page with a text area for sending text to a JSON-RPC service.

from pyjamas.ui.RootPanel import RootPanel
from pyjamas.ui.TextArea import TextArea
from pyjamas.ui.Label import Label
from pyjamas.ui.Button import Button
from pyjamas.ui.VerticalPanel import VerticalPanel
from pyjamas.JSONService import JSONProxy


class JSONExample:
    def onModuleLoad(self):
        self.rpc_service = JSONProxy('/cyborg/default/call/jsonrpc', ['myfun'])

        self.text_area = TextArea()
        self.text_area.setText(r"Hello World")
        self.text_area.setCharacterWidth(80)
        self.text_area.setVisibleLines(4)

        button = Button('Send to Server', self)

        self.status = Label()

        panel = VerticalPanel()
        panel.add(self.text_area)
        panel.add(button)
        panel.add(self.status)

        RootPanel().add(panel)


    def onClick(self, sender):
        print('sending to server')
        self.status.setText('Waiting for response...')

        textarea_txt = self.text_area.getText()
        if self.rpc_service.myfun(textarea_txt, self) < 0:
            self.status.setText('Server Error')


    def onRemoteResponse(self, response, request_info):
        print('good response')
        self.status.setText(response)


    def onRemoteError(self, code, message, request_info):
        print('error')
        print(code)
        print(message)
        self.status.setText("Server Error or Invalid Response: ERROR  - " + str(message))


if __name__ == '__main__':
    app = JSONExample()
    app.onModuleLoad()

To make the call, you need an instance of the JSONProxy class, so I have the line

self.rpc_service = JSONProxy('/cyborg/default/call/jsonrpc', ['myfun'])

Then call your function using that instance, along with passing in an object/variable (which Pyjs encodes) to send to the service, and a reference to an instance of a class with onRemoteResponse and onRemoteError methods.

self.rpc_service.myfun(data, response_class)

In the example code above, response_class is self, so the JSON response comes through onRemoteResponse.

Putting it Together

Believe it or not, this was the trickiest part for me when I first got this working. Figuring out the correct URIs to use, and having code listening for the call was tough to debug. Luckily for anyone following along, the example code here already has everything setup correctly :).

We can start out just by making sure everything ties together. Go to your web2py folder then applications\cyborg\static. This is the static directory for the cyborg app. Now take all the output from Pyjs and put it there. I called my Pyjs file JSONExample.py, which generated JSONExample.html, so I access this using http://127.0.0.1:8000/cyborg/static/JSONExample.html. The ‘Send to Server’ button on the page should work. If it doesn’t, make sure you followed every step exactly. There might have also been changes/bugs in Pyjs or web2py – it’s not likely, but it has happened.

I personally wanted my main app to call the JSON-RPC service, not a file in the static director. I am going to have the index load the JSON page. First thing to do is cleanup the index function in default.py. I just want index to return the view, so index now returns an empty dictionary:

def index():
    return {}

I then replace the contents of applications\cyborg\views\default\index (the view for index() in the default controller) with the contents of the main Pyjs output file, JSONExample.html in my case.

The compiled html/js files are still in the static dir though. Remember, users are accessing the files in the views directory by accessing controllers. So if the other Pyjs files were in the views, you would still need to have a controller function to access them. There is no clean, simple way to put all your Pyjs cache files in the views directory. Instead we need to modify the new index.html to point to the files in the static dir. So the index.html code gets modified once for the module:

<meta name="pygwt:module" content="/cyborg/static/JSONExample">

and, in several places, for bootstrap.js:

<script language="javascript" src="/cyborg/static/bootstrap.js"></script>

Now you can visit http://127.0.0.1:8000/cyborg/ and send JSON-RPC calls!

One thing to note is that with the default routes calling /cyborg/, /cyborg/default, and /cyborg/default/index all load the same view/controller. If your JSONProxy class uses a relative link it might only work when visiting 1 of these URIs. That is why, in the Pyjs code, I refer to URI starting from /. When I was learning how to do this, I set everything using relative URI’s, like ../static/JSONExample, and ../default/call/jsonrpc, and that made everything difficult, so I stay away from that.

A Simple web2py App

I know this is waaaaay overdue, but better late than never. Over the last year and a half I’ve switch over to using Linux and away from Pyjamas (I’m still using python though). I’ll have more info in future posts over the next couple weeks. Don’t worry if you’re using Windows though, the steps here work in both Linux and Windows (thank you Python!).

Installing and Running web2py

Start by downloading web2py (source – version 2.2.1 for me). I am doing this in XP, so I extracted the code to C:\Projects\web2py. Open a command window, navigate to your web2py dir and start it up with the command

> python web2py.py -a tmp_pass

This starts up web2py on 127.0.0.1:8000 with the admin password set to tmp_pass. You can use the -h option to see how to set web2py up in other ways. One thing to note is if the server is running on 127.0.0.1 you won’t be able to access it using your real IP address. If you want to test your server using external computer have web2py use the IP 0.0.0.0.

With web2py running, I could then visit http://127.0.0.1:8000 where an sort of hidden admin interface button lets me login using my admin password, tmp_pass.

Creating an App

There is a panel on the right, with a section called “New simple application” which you can use to create an app. This sets up all the template files for you. In my case I created a program called cyborg.

The server shows a list of files which I could edit. It’s a lot of code, definately more than I wanted for my app, but that should be easy to cleanup later on. With the web2py server up, I navigated to http://127.0.0.1:8000/cyborg/. which showed a page with some interesting bullets, including:

  • You visited the url /cyborg/
  • Which called the function index() located in the file web2py/applications/cyborg/controllers/default.py
  • The output of the file is a dictionary that was rendered by the view web2py/applications/cyborg/views/default/index.html

These are the MVC files discussed in my previous post.

Simplifying the App

I decided to investigate each of the steps taken to run the code and try to trim unneeded code: 1) Well, this one is obvious

2) The default code has calls to use databases, and uploading files, etc. Actions I don’t plan on supporting, at least just yet. First, I want to get comfortable with everyting. I changed default.py to be the simplest function possible:

# -*- coding: utf-8 -*-

def index():
    return dict(message="Hello World", message2="How's it going?")

3) The view should get the dictionary from index() and render it. Really whats happening is the view is loaded, and any python code in the index uses the dictionary from index(). Python code is embedded between {{}}

{{if 'message' in globals():}}
<h3>{{=message}}</h3>
{{pass}}
<br>
{{if 'message' in globals():}}
<h3>{{=message2}}</h3>
{{pass}}

The only kind of gotcha I found was that you need to have a pass aligned with every if. I think this is because there isn’t a way to unindent in the html files.

Cleanup

Since I only plan on using a couple function to start, I want to remove all the files that seem unnecessary. I started by removing and checking the page still worked: I removed these folders completely (I think most get recreated by web2py when the app runs, but with just dummy files):

  • databases (this folder has the database files)
  • errors (this is a logs folder)
  • languages (translation files)
  • models (usually where you put information about your database)
  • private
  • static
  • uploads

And cleaned the folders:

controllers: Only kept default.py views: only kept views\default\index.html

I realoaded http://127.0.0.1:8000/cyborg/, and I was happy to see the messages I passed in through the dict.

Using Web2py

As promised in the GAE post, here is part 1 of using JSONRPC calls on GAE. I originally intended to write more of a how-to, but now that I’m halfway through, this seems like it should be it’s own post, and I’ll write a how-to part later.

If you’re reading this blog, you probably know how to write Pyjamas apps, but you might not know about web2py, and why or how to use it. I will give a quick overview on how web2py works in the 2nd half of this post, but if you want to just know about using web2py with Pyjamas, or deploying to GAE (on windows), skip to the next post.

I want to start out with a warning. You MUST be careful if you plan to develop an app on GAE. Let’s say hypothetically you have an idea for a business using webapps. You start out with no money and no users. GAE looks really appealing because you can get reliable hosting for free. A few months later you’ve grown and you’re ready to move to your own server. This is where you run into issues. All the server code was written for GAE. If you were good and your code is clean, you can copy several large chunks of code directly over to your new server, but interfaces for things like JSON calls, and interfaces to your database will all need to be rewritten. If you look around, you can find a lot of talk about “portability” and “lock in” and ways to get around this issue. It’s definately a hot topic when it comes to using GAE.

Lock in only happens if you aren’t careful up front. You don’t have to develop using GAE directly, instead I suggest using a web application framework, and build your code on top of that. At the very least, this will protect your code. I’m working on a small game, so I haven’t looked at all into moving database info, but this is something you might want to research. Worst case, you could write a web app to print out all your db info on a web page in a format that you can import to your new server. If anyone knows any good ways to move databases, feel free to leave a comment!

Back to frameworks. There are a couple frameworks available that support GAE. The two most popular are web2py and django. From what I read, neither framework is clearly better – each has its own strenths. It’s more about finding a framework that fits your needs more. For hobbyists, I’d recommend web2py. Web2py supports quicker development. A lot of people say it’s more intuitive and you can be more productive. I feel it aligns better with the Python philosophy. Django has it’s own strengths though. It gives you more control as a developer. Sometimes you’ll want that extra control, it’s really up to you.

If you’ve decided to go with web2py, or even if you didn’t, here is a quick overview of how it works from my non-SW perspective. There are 4 areas I heavily use inside each of my applications – model, controller, view, and the static dir. With that in mind, I want to start with the web2py URLs which take on the form of http://domain/application/controller/function (where function is closely related to view). You also have a static directory which can be accessed using http://domain/application/static/. The static dir is where you’ll have website images, Pyjamas apps, and other static files. There are some defaults setup so if someone visits http://domain/ your server will load something, but most URLs will take on the form http://domain/application/controller/function.

So what do controllers and functions do, and how are they related to views? Well you have a “controllers” folder in your application’s directory which is full of controller scripts made up of several functions. If someone visits http://domain/application/game/loadGame the server will call the game.py file in your controllers directory and run the loadGame() function in that file.

This leads to views. Some functions will be used to handle forms, or handle JSON calls, but your most important functions will return a dict(). When you return a dictionary, it is really returning a reference to the view with the same name as the function. Let’s say this loadGame() function returns a dictionary. In your views directory there should be a html file game/loadGame.html. This is what the server sends users when they visit http://domain/application/game/loadGame.

This views page will probably include links to js. files, images and other static files. This is the 3rd area, the static dir. When you have static files, you will host them here.

There’s one last area – models. The models area has the setup for any databases you’re using, and the setup for other things like the JSONRPC interface. When I tried adding users and storing high scores, I was using this file a lot.

So that’s web2py in a nutshell.