The good, the bad and the Zope

I want to use Zope 3 interface package to write component architecture i.e. have a plug-ins easily in Python. Zope 3 interfaces are very handy and, which cannot be conducted from the name, are available outside Zope too. From my prior experiences I know that Zope 3 interfaces package is one of the best and most underrated Python packages out there. It even influenced to the new design of Python 3k.

Well then… I haven’t used Zope 3 interfaces standalone before, so the first thing what I do is writing “zope 3 interfaces” into my Google search this.

This page comes up.

It’s horrible - the very reason I write this quick blog entry. Some notes below (I have written things from the point of external visitor - I have hands deep in Zope myself, so you don’t need to clarify these things for me or teach anything)

In the post “No, you are not smart enough for Zope” Martjin Faassen highlights some problems of Zope community. “It’s hard to get good content written” Martjin claims. I disagree. Whoever created the page originally could have thought what people coming to the page want. They don’t want to decrypt the brain core dump of hardcore Zope developer. They want to know what is this thing, how this thing is beneficial for them, how do I get started with it and how do I use it.

You all know how Internet works. You all have visit on web pages. You all are customers for the same thing you also produce. So writing a basic web page is not something you couldn’t do.

Hints:

Pardon me the tone of this post. Zope is the 23th best thing out there, but the Zope community has stagnated badly in some aspects. Some things were acceptable ten years ago when web was still young and Python developers hardcore, but if you don’t keep with the pace you lose all the mindshare.

Verifying Javascript files with JSLint from Eclipse IDE

With Web 2.0, Javascript has gained a great foothold in the web development. Javascript, which was originally indented to be few lines of interactivity here and there, is now used for full-blown UI frameworks like Ext JS and Dojo. Because its toy background, Javascript is not the easiest language to debug. Also, the lovely phenomenon known as browser wars has ensured that professional Javascript development is PITA due to browser incompatibilities - It seconds the Symbian C++ embedded programming if you know what I mean.

Javascript is executed run-time. Thus, any errors cannot be cached until the code is run. JSLint is a verifier tool which checks Javascript syntax offline. Of course, it cannot detect application logic errors, but it makes sure that you don’t have cross-browser compability problems in your Javascript syntax. Maybe the most famous of these problems is the extra comma before the list termination which is ok for Firefox, but kills the page on Internet Explorer.

Some argue that this does not really catch bugs and instead a comprehensive unit-test suite should be used. This might be a requirement for a platform level Javascript library code, but it is often an overkill for your little site with some flashy dialog windows.

Installing JSLint

JSLint itself is written in Javascript. So you need to offline Javascript interpreter to run it. On Linux there exists Rhino. On Ubuntu, Rhino package is broken. and you need to download the orignal Rhino JAR from here and copy it to /usr/share/java.

sudo apt-get install rhino # Note https://bugs.launchpad.net/ubuntu/+source/rhino/+bug/93885

Then get JSLint offline version from here.

Running JSLint straight from the Eclipse

We use JSEclipse plug-in in Eclipse for Javascript development. Thus, it is convenient to to execute JSLint directly from the Eclipse IDE.

Drag and drop jslint.js to your project/scripts folder in Eclipse.

Create a new External tool like in the example below.

screenshot-external-tools.png

Then you can run it by just clicking “run external tool” icon. Output comes to the console window.

screenshot-pydev-nurse-source-server-media-js-ui-mapjs-eclipse-sdk.png

Checking all Javascript files during unit test

verify_javascript.bash can be executed from shell and it checks all JS files in your project. This is a nice feature to integrate to your unit test cycle.

Example:

$ bash scripts/verify_javascript.bash source/server/media/jssource/server/media/js/jquery.js

Lint at line 11 character 30: Expected '{' and instead saw 'var'.

...

Symbian digging its own grave with symbiansigned.com

Edit: Made correction to iPhone SDK price/added self-signed option

I have been recently involved in Symbian development. Unfortunately, I could say. Symbian has made the life of Symbian developers headacheful. The main problem is that the application signing is made unnecessary cumbersome. Getting a required developer certificate to just to run your one liner test code is a long road paved with curses, stress and ponder how a world class company can screw up things so badly.

I write this post to vent out the frustration. Hopefully someone picks up the feedback presented here. Personally, I don’t care anymore really. After explaining to the customer once “I am afraid that we cannot have the demo in this meeting, since we were unable to get the developer certificates in time. It was nice you did summon all of your executes, though.” my sympathy towards Symbian has fallen to zero.

In the long run, I believe, this will hurt Symbian when developers leave the platform. When this happens Symbian will miss 1) developer base 2) the wow value of all new cool things when hackers move to happier platforms - after all you don’t want to work with things causing a headache even if you are paid generously. Professional Symbian capable developers are already rare and scaring away the potential candidates (students, open source developers, freeware developers) won’t really help the matter. Of course, for us, who are already in the business, it’s good since we can raise our hourly fee and die as rich dinosaurs (remember COBOL).

Looks like the root problem is the cocky attitude: “We are the biggest mobile operating system in the world. You are not our customer, the biggest mobile phone brands are. Thus, you are an insignicant fly and we don’t need to listen to you. Please go away.” The big boys tell that this very same reason lead to Motorola’s fall from the number one mobile phone vendor position - so let’s see if the history is repeating itself.

The problems of Symbian development process, especially signing, are well known. Symbian Signed forum category has 120 000 views. Symbian signed support requests has 57 000 views and 2000 messages, since November 2007. Over a thousand support requests in four months. Symbian has money. It could fix the problem if it has will. If there is will in the organization and the work just does not progress, the corporate world has a simple solution: fire the management and hire capable responsible persons.

I won’t go the details of the problems in this post. I have prepared a post containing over 50 bulletins with detailed steps to get the developer certification process done and it lists every little mistake in Symbian Signed process. Unsurprisingly, the post mentions fuck and hell over twenty times, so I am reluctant to post it on our company site. So I sketched a manager comprehensible overview of the state of the process in this post. To justify my criticism, I compare Symbian to Apple iPhone and make some suggestion how to fix things.

Certificate costs

Symbian: 200$/1.5 years (you need a publisher id to get a developer certificate) + 20$ per signing round

Update: You can still make self-signing certificates yourself with limited capabilities (no GPS), but symbiansigned.com doesn’t bother to mention it.

Apple: 99$/year (for testing and distribution applications)

Suggestion: Make the developer certificate free. It used to be free, but based on this post, Symbian signed claims that they do not have enough CPU power, because of all kind of nasties are knocking their server. Come on! it’s Internet. You are hosted by Cidercone and by knowing how much they charge for the hosting I believe you can get every possible server power in the world to generate your keys.

This development start up cost automatically excludes third world people, freeware makers and students i.e. the future Symbian professionals. It’s hard to maintain a platform if there is not enough skilled people to work with it. There won’t be meat into Nokia’s and Sony-Ericsson’s innovation machines.

Development start lead time

How long it takes to get “Hello World” code to the phone.

Symbian: 8 workdays

Apple: 6 hours

Complete Guide to Symbian Signed guide (manual) is 20 pages PDF. How often you need a user manual to use a web site? Even with this page amount, the manual lacks crucial pieces of information which you need to hunt and ask help from the Symbian Signed support forums. I had to make 3 support request for Symbian Signed and 1 for TrustCenter.

Set-up process

Symbian: Create 3 user accounts (symbiansigned.com, developers.symbian.com, TrustCenter). Fax your passport and some signed papers to TrustCenter Germany. Wait when your request is manually processed. Ask TrustCenter what I do with the link they send me (hint: the only working web browser for their site is the very same browser you orignally submitted your application from). Convert TrustCenter key to Symbian specific format using obscure BAT tool. Use half-baked Windows EXE to generate files. Ask at the support forums permission to be allowed to create the Publisher id.

Apple: Download SDK. Click “I Agree.”

Symbiansigned management team must be either incomplete or inexistent, or they just hate their users. Looks like no one didn’t bother model or test this process, since the flaws are so apparent. Why do we need three user accounts? Why do we need a user account at all, isn’t my credit card payment enough? Why there can’t be a single web page on symbiansigned.com where I could do these all things without? Also, Symbian could do these things without third parties (TrustCenter) who are badly integrated to the process.

Web site and information reachability

Symbian: Two sites with different user accounts. Information is spread around and hidden behind various for registred users only pages. Often Symbian PDFs (yes they seem to prefer zipped PDFs) and web pages cross-link to other sites and these links are broken. Their internal site search engine does not give meaningful results. Google cannot search pages, since they are hidden behind the registration. Web forms don’t work. Navigation paths are unclear and you often need to guess where a certain piece of information could be.

Apple: One site - and it just works.

Why Symbian hides a crucial BAT tool in the page labyrinth of developers.symbian.com and behind login? Does the 100 lines of BAT code contains a very hidden trade secret? I doubt. (I also doubt that a BAT tool is a good way to solve a problem which could be fixed with a web form or Javascript, but that’s another matter.)

I think this is the part of the attidude problem. SymbianSigned.com mission is not to “secure” the development and control the developers of the platform. It’s mission should be easily enable developers to start development and release their software for Symbian. The meaning of “easily” includes that the web site is working and up to the standards. Please hire a usability expert (we are here) and get it fixed. This problem is not solvable by writing yet-another user manual PDF.

Giving feedback

Symbian: Support like goes to the Symbian developer forum. A bulleting board with unorganized, angry, posts. They don’t give a feedback form to report the problems on Symbian Signed.

Apple: “There are three great ways to get answers to your questions about ADC membership, products, and services: visit our Frequently Asked Questions web pages, submit your questions or feedback using the form below, or call us to speak directly to a worldwide support agent. We look forward to assisting you.”

I’d help symbiansigned.com to get its site better if I could. When there is a broken link, I can hit feedback and ask them to fix it. But looks like they want to be ignorant about the matter - if I report a problem on their forum I bet no one will ever fix it. There won’t be even “thank you.” Probably no one ever reads my post. On the other hand, when having such non-functional service, this feedback channel would be flooded in overnight.

Conclusion

Symbian has unforgiveable screwed up with its signing process and doesn’t seem to get the feedback from the developers who suffer from it. Steve Ballmer’s famous quote “developers, developers, developers” refer to what’s the important factor making the platform succesfully in long term. This is especially crude when you know that things used to be good.

SDK released - Python in iPhone?

I just read waffle’s blog entry about iPhone SDK release. Looks like Objective C is the only supported language by default (I am just downloading SDK).The comments speculated that embedding Python is not possible due to size constraints. Bollocks I say =) Python for Series 60 phones is 500 kb download without trimming. It’s less than the size of HTML page you are viewing now - RAM footprint is even smaller) If Series 60 phones, which have much more modest hardware specifications, can run Python it shouldn’t be a problem for iPhones either.

Why Apple didn’t add additional language support by default? Well they seem to have their hands full to get SDK out at all (delays) so we shouldn’t expect to have perfect set in 1.0 release.

Now, who wants start a porting project with me? ;)

Debugging Django memory leak with TrackRefs and Guppy

I run Django in a standalone long-running application (video encoding server). It leaked memory severely. By using htop, one was seeing two gigabytes reserved for /usr/bin/python after a while. Before starting the debugging session, I had no faintest idea what could be the cause of the problem. Django is robust technology - this kind of things haven’t happened for me before. Since I was running Django in standalone mode, I suspected that some query cache does not get cleared. But random poking around the source code didn’t give any clues.

It was time to do some serious memory debugging for Python.

Python as is doesn’t leak memory, since it’s garbage collected virtual machine. All “leaks” are design problems in the application logic.I found a good primer here what’s going inside Python’s memory management.

First I tried this nice TrackRefs class from Zope. It relies on Python’s own in-interpreter functions to monitor objects.

class TrackRefs:
    """Object to track reference counts across test runs."""

    def __init__(self, limit=40):
        self.type2count = {}
        self.type2all = {}
        self.limit = limit

    def update(self):
        obs = sys.getobjects(0)
        type2count = {}
        type2all = {}
        for o in obs:
            all = sys.getrefcount(o)

            if type(o) is str and o == '<dummy key>':
                # avoid dictionary madness
                continue
            t = type(o)
            if t in type2count:
                type2count[t] += 1
                type2all[t] += all
            else:
                type2count[t] = 1
                type2all[t] = all

        ct = [(type2count[t] - self.type2count.get(t, 0),
               type2all[t] - self.type2all.get(t, 0),
               t)
              for t in type2count.iterkeys()]
        ct.sort()
        ct.reverse()
        printed = False

        logger.debug("----------------------")
        logger.debug("Memory profiling")
        i = 0
        for delta1, delta2, t in ct:
            if delta1 or delta2:
                if not printed:
                    logger.debug("%-55s %8s %8s" % ('', 'insts', 'refs'))
                    printed = True

                logger.debug("%-55s %8d %8d" % (t, delta1, delta2))

                i += 1
                if i >= self.limit:
                    break 

        self.type2count = type2count
        self.type2all = type2all

You need to have Python compiled in debug mode to have sys.getobjects() method. Luckily this beefed up Python binary is availalble from Ubuntu’s stock repository:

sudo apt-get install python-dbg python-mysqldb-dbg

Note that native Python extensions don’t work unless they are specifically compiled against the Python debug build (python-mysqldb-dbg)..

Then I add TrackRefs to my main loop:

    def run(self):

        self.running = True

        logger.info("Started worker " + self.get_worker_id_string())

        # Memory leak tracking
        tracker = TrackRefs()

        while self.running:

            self.mark_for_download()

            self.process_downloads()

            self.process_encodings()

            tracker.update() # Dump memory here
            time.sleep(settings.WORKER_POLL_DELAY)

And after running a while I start getting interesting results:

7956 [2008-03-07 02:59:28,767] INFO Jobs needing sources to download 0
7956 [2008-03-07 02:59:28,768] DEBUG Processable jobs: 0
7956 [2008-03-07 02:59:29,754] DEBUG ----------------------
7956 [2008-03-07 02:59:29,754] DEBUG Memory profiling
7956 [2008-03-07 02:59:29,754] DEBUG                                                            insts     refs
7956 [2008-03-07 02:59:29,754] DEBUG <type 'int'>                                                 150   137406
7956 [2008-03-07 02:59:29,755] DEBUG <type 'tuple'>                                               117   130211
7956 [2008-03-07 02:59:29,755] DEBUG <type 'dict'>                                                  5     8331
7956 [2008-03-07 02:59:29,755] DEBUG <type 'str'>                                                   3    27643
7956 [2008-03-07 02:59:29,755] DEBUG <type 'unicode'>                                               3     4606
7956 [2008-03-07 02:59:29,755] DEBUG <type 'list'>                                                  3     3492
7956 [2008-03-07 02:59:29,756] DEBUG <type 'frame'>                                                 1      962
7956 [2008-03-07 02:59:29,756] DEBUG <type 'cell'>                                                  0    12948
7956 [2008-03-07 02:59:29,756] DEBUG <type 'function'>                                              0     9479

Woah! Who reserved 130 000 ints and tuples? No wonder that soon python gulps 1 gigabytes of memory. Since this is the only number which grows during the main loop cycling and there is no references to classes or objects debugging becomes a bit more difficult. I need to try to cross-reference the difficult tuple objects.

This didn’t go well - with gc.get_referrers() recurive parsing I got some results (example below). But it became soom clearthat debug references from the system itself was impossible: the memory debugging code will always create nasty cyclic references to the system, since it needs to track the objects. I gave up. There had to be something better.

9154 [2008-03-07 04:05:23,571] DEBUG /var/lib/python-support/python2.5/MySQLdb/connections.pyc
9154 [2008-03-07 04:05:23,571] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,572] DEBUG defaulterrorhandler
9154 [2008-03-07 04:05:23,572] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,572] DEBUG string_literal
9154 [2008-03-07 04:05:23,572] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,573] DEBUG unicode_literal
9154 [2008-03-07 04:05:23,573] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,573] DEBUG string_decoder
9154 [2008-03-07 04:05:23,573] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,573] DEBUG __exit__
9154 [2008-03-07 04:05:23,574] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,574] DEBUG begin
9154 [2008-03-07 04:05:23,574] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,574] DEBUG __init__
9154 [2008-03-07 04:05:23,574] DEBUG <type 'function'>
9154 [2008-03-07 04:05:23,575] DEBUG show_warnings

There was: Guppy. Thank you Sverker Nilsson! You saved my day.

Since the API of Guppy is a little eccentric, here are some examples for you:

# init heapy
heapy = guppy.hpy()

# Print memory statistics
def update():
  print heapy.heap()

# Print relative memory consumption since last sycle
def update():
   print heapy.heap()
   heapy.setref()

# Print relative memory consumption w/heap traversing
def update()
    print heapy.heap().get_rp(40)
    heapy.setref()

With heapy.heap() ; heapy.setref() I got this output:

Partition of a set of 12 objects. Total size = 3544 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      3  25     2244  63      2244  63 unicode
     1      2  17      708  20      2952  83 types.FrameType
     2      3  25      432  12      3384  95 dict (no owner)
     3      3  25      120   3      3504  99 str
     4      1   8       40   1      3544 100 list

One adds get_rp() travelsal magic and everything becomes clear:

Reference Pattern by <[dict of] class>.
 0: _ --- [-] 14 (dict (no owner) | list | str | types.FrameType | types.Gene...
 1: a      [-] 3 dict (no owner): 0x8c11f34*2, 0x8c1bd54*2, 0x8c1f854*2
 2: aa ---- [-] 1 list: 0x833c504*18
 3: a3       [-] 1 dict of django.db.backends.mysql.base.DatabaseWrapper: 0x8...
 4: a4 ------ [-] 1 dict (no owner): 0x83a65d4*2
 5: a5         [R] 1 guppy.heapy.heapyc.RootStateType: 0xb787c7a8L
 6: a3b ----- [-] 1 django.db.backends.mysql.base.DatabaseWrapper: 0x8356a34
 7: a3ba       [S] 7 dict of module: ..db, ..models, ..query, ..transaction...
 8: b ---- [S] 1 types.FrameType: <<lambda> at 0x8b16ecc>
 9: c      [-] 2 list: 0x833c504*18, 0xb7dafe6cL*5
<Type e.g. '_.more' for more.>

What there could in DatabaseWrapper object which is growing and growing… query debugger. Django keeps track of all queries for debugging purposes (connection.queries). This list is reseted at the end of HTTP request. But in standalone mode, there are no requests. So you need to manually reset to queries list after each working cycle.

        while self.running:

            self.mark_for_download()

            self.process_downloads()

            self.process_encodings()

            tracker.update()

            time.sleep(settings.WORKER_POLL_DELAY)

            # Clear database connection ad reset query debugger
            # between cycles to make sure that
            # related resources get released
            reset_queries()
            connection.close()

            print str(connection.queries)

But even after this fix, I got increase in tuple and int usage when monitoring with TrackRefs. But when I run heapy.heap() alone, there is no increase. So the tuple and int consumption must have been caused by TrackRef, sys.getobjects, gc, etc. magic itself.

Building commercial grade mobile applications with Python for Series 60

Python programming language, created in 1991 by Guido van Rossum, is the sixth most popular programming language and “the programming language of year 2007“. Nokia Series 60 is a Symbian-based mobile phone platform for high end smart phones which is the most widespread smart phone platform. Python for Series 60 is Python ported to S60. PyS60 is open source and there already exists a notable community around it - a sign of a successful software platform.

This essay is a post-mortem for creating a commercial grade PyS60 application. By “commercial” I mean something which is publicly available for end users and meets the criteria of Symbian Signed certification. I hope this essay will shed light on the real world challenges, thus aiding Nokia, mobile application vendors and PyS60 community to know when to use PyS60, to strengthen PyS60 as a platform and to guide people make most of it.

Description of the project
I was given a task to create a mobile phone client for a web service. The client runs on the the background, posting information from the phone (files, location) to the web site. Two developers were assigned for the job. In the end of the project, almost 100 Python modules and classes had been written (just for the mobile side of the things). I believe this is the most complex PyS60 development effort this far.

The application is divided in two parts: user interface process and background process. The background process is running all the time, monitoring the phone state and posting changes to the server. The UI process can be launched from the phone menu and it commands the background process. Two processes communicate over localhost TCP/IP sockets. The application communicates with the Internet server by HTTP multipart posts. The background daemon had four threads running: main thread, server communication thread, interprocess communication thread and GPS thread.

The core modules of the application were written portability in mind. In fact, one can run the core as standalone Python application on a normal PC. This makes unit testing and debugging much easier. Some supporting libraries and files were ported from the normal Python 2.2.2 distritbution to PyS60.

Python for Series 60 and the alternative platforms

When we started the project, I was not aware of commercial applications built on PyS60, besides Nokia’s own Mobile Web Server. I knew it was used as in-house in few large software companies. Python has traditionally had strong foothold in the academic world. So it does not come with a surprise that Finnish universities use PyS60 on mobile application software development courses due to Python’s low learning curve.

Python itself has been around longer than the most popular programming language, Java. Python’s suitability for the complex development jobs was largely untested until the rise of modern web. The dynamic, agile, nature of Python does not come for free. So called abstraction penalty (interpreting) slows down Python execution speed. It was not until the 00s when processor were fast enough to make this irrelevant matter: software development costs outrun hardware costs and more agile development tools had to be found.

When building S60 applications, you need to chose the progamming language between three candidates: Good old fashioned C++, Java Micro Edition (Java ME) and Python for Series 60.

Currently, most of Series 60, and Symbian, platform specific applications are written in C++, with a twist. One cannot consider this as “normal” C++ code, but the bastard child of this language. In their quest for the ultimate embedded efficiency (read: the smallest memory footprint) Symbian architectures placed an enormous mental burden on the poor Symbian developers. For example, Symbian C++ has custom pointer schemes, manual error handling, wicked threading and a weird build process. Or to put it bluntly: writing C++ for Symbian is far more difficult than writing C++ for platform X. You cannot really use any standard libraries on Symbian (this is changing, however). Thus, building S60 applications based on C++ technologies is costly, and painful. I have not yet met a happy Symbian C++ developer and I have met hundreds of them.

Then, we have Java ME. Sun made a good move by creating a unified mobile application run-time environment. Java ME is the only way to get your mobile application running regardless of the mobile phone vendor. But Java’s infamous slogan “write once, run everywhere” doesn’t hold a candle.Java ME implementations have internal differences and they are buggy as two weeks dead squirrel (seen one squirrel once and followed Sun’s Java ME mailing list long enough). Java ME gets the new features last due to standardization process. Java ME applications are heavily sandboxed and they cannot access half of the phone features.

And as the last option is Python for Series 60. Started as a Nokia’s research effort, Nokia brings this agile environment to mobile software development. Python for Series 60 has also a sister project, PyUIQ, which targets Sony-Ericsson phones.

PyS60 has had the reputation to be “prototype only” technology and it’s use has been counter-argumeted by “serious development needs C++”. This is slowly changing. Nokia has shown serious commitment to Python. With the launch of Mobile Web Server, Nokia proved that Python is feasible in commercial grade applications.

This is how my brain worked when I chose PyS60: Symbian C++ is notorious difficult and no sane person wants to ever touch it. Java ME might have not enough access to the phone features to fulfill the application requirements My client already used Python on embedded Linux and web servers. Why not give PyS60 a shot?

Arguments for Python for Series 60

Python is agile

This is maybe the most important argument to pick PyS60. Python tends to reduce development time compared to C++ or Java. The language itself is very easy to learn. Syntax is expressive and developers can achieve much more functionality in less lines of codes: You can focus on getting the task itself done. With higher level concepts and less hassle with lower level things, changes to introduce bugs radically decreases.

Python development does not have traditional compiling phase, though byte code files can be built for the final release to obfuscate the code and optimize loading times. Changes made into Python code are immediately (in seconds) visible in the running application. Even if the application runs on the target device, since code changes can be automatically synchronized over a Bluetooth connection. No wonder Python enjoys legendary fast prototyping language fame - the developers have less time to surf in web when they don’t need to wait compilers to bake yet another test build.

PyS60 is open source

I assume you are familiar with the benefits of open source already: vendor freedom, no licensing cost, not being mercy on others and so on. This all holds true for PyS60 as well.

PyS60 license (Python license) does not place restrictions for creating closed source software. You can have very good development tools for free: Series 60 SDK, Eclipse IDE with PyDev plug-in and PUTools bluetooth synchronization kit for free. This makes the barrier to enter PyS60 development very low.

Since PyS60 is fully available in the source code form, you can customize and fix it for your needs. The traditional horror story from Java ME development is that your application won’t run in the mobile phone model X, since there is a bug in the vendor’s Java Virtual Machine implementation. Unless you are A Very Big Player, don’t ever think about going to talk with the vendor and ask it to fix its bug - hitting one of these issues might prevent you to release product at all. With PyS60 you can control all aspects of the software execution up to the closed parts of the device. No more being mercy on others!

Being open is pop now. Embracing open source gives much for your company: good citizen reputation, hiring pool and free marketing.

Python is portable

Python runs on Linux, Windows mobile and Symbian - Python runs on almost every smart phone known to man. And not just the phones, it runs on every computer as well: Windows, OSX and Linux server and desktops. Moving your code between these environments is easy. Maybe there isn’t yet “cross-platform Python SDK for mobile phones”, but it might be just a matter of time when something leveraging the idea comes out.

In our project, core files, independent from graphical user interface, were built and unit tested on Linux desktop environment. No emulator was needed. It was not until the late development stages when they had to be moved to Symbian environment. Since you can develop and debug your code in such “easy” environment, you can work more effectively.

Because the full Python is available in the phone and it can run the exactly same modules as your servers, sharing code between these two environments opens possibility for making “smart” clients. Think about the situation where your phone is offline. It can have a subset of the full database loaded on a memory card. Since moving Python code from the server to the mobile is easy, the mobile application can have the same database functions and application logic modules loaded with it. Without the extra duplication of the logic code, one can operate on the small subset of the data and later synchronize it with the main database.

Python is extendable

You can mix C++ and Python. When you run out of ready components or you need unleash all available CPU cycles for CPU sensitive calculation tasks, you can write your own native extensions (DLL files) for PyS60 in Symbian C++. They enjoy all the same privileges as any native built code or application. Actually standalone PyS60 applications are just normal Symbian executables which just happen to have Python interpreter started inside them.

If you are working with phone features which are not yet very common and lack working Java ME implementations, say acceleration sensors, instead of creating a native C++ application you can create a Python application. You need to mix-in low level C++ only for the the critical part of the applications which operate with the native platform API.

PyS60 community is vivid

Surf to forum.nokia.com Python discussion boards and you can witness the activity around PyS60. A lot of folks are answering the beginners’ questions and guiding them. If your case is not totally unique, you can expect help to be found. As a matter of fact, you, reading this text, is a sole proof of powerful community. I was not directly paid to write this and I am not affiliated with Nokia any way - this writing flows from the bottom of my heart to promote PyS60 and open some window for the (hireable) talents of our company.

You love it

For some things, we just feel impulsive attachment. They are beautiful, aesthetic, sexy. When you see it, you need to have it. PyS60 is one of these things.

Arguments against Python for Series 60

PyS60 has not been around for long

PyS60 is a young project. It still has some maturity issues. If you don’t do preparations well and test that all needed modules work as you want them to work, you might need to do some extra work to get around the issues.

Especially PyS60 user interface lacks some crucial components and polish which may force you to create UI components yourself.

By “getting around” the issues I mean leaping to the native Symbian code world and writing embedded C++. Since this leap is all but a little step, in the terms of development skills and complexity, I recommend preparing for the worst before the roof is falling on your head. I have seen too much “plz reply me asap urgent my boss kills me this afternoon” PyS60 discussion board messages from persons who haven’t done their homework. For example, you could make sure that you have some hired Symbian guns available in the case your own effort was not enough.

Python is for high-end phones only

PyS60 and PyUIQ are available for the high end smart phones only. Nokia’s Series 40 devices, and other devices built around in-house operating systems, cannot run Python. If you are targetting to e.g. consumers markets and Java ME has all the required capabilities to implement the application, it is the only real way to go.

Symbian is dreadful platform

Symbian, as a software development platform, is very challenging. Due to its history, the feeling of developing for Symbian differs greatly from other mobile, desktop and server platforms. Things simple elsewhere, like installing an application, might become a development nightmare on Symbian where one has to fight his/her way through capability limitations, artificial device restrictions and signing process just to get the application to the phone.

PyS60 naturally inherits all these challenges. Java ME goes around the issue by having its own custom installation, signing, etc. schemes. On Symbian, even accessing some of the device features - a device you own and hold in your hand - requires you to throw in few hundred bucks cash to get the publisher id - to get the developer certificate - to sign the application - to have all required capabilities - to access the specific features (read: GPS). More cash and third party testing is needed to publish the application. This is the price of virus free DRM’ed platform.

Lack of entrepreneurship spirit

If your company states “we are a Java house” or “we are a C++ house” and you miss courage to look for alternatives, then maybe you shouldn’t. Sailing into the brave new world takes more resources than staying near your home harbor. On the other hand, if you are a technology company, it’s quite important follow technologies outside your mainstream, unless you want to suffer the extinction when the new kids on the block gain power (remember the day when COBOL died).

Conclusion

Nokia has done a great deed and effort by creating PyS60 and then giving it out for us to play. Now its own turn to act as a community to give back something we can.

Building commercial grade PyS60 applications might not be straightforward yet, but it is possible. Working with PyS60 is definitely more productive than using complex Symbian C++. It is not always straightforward as Java ME, but eventually it will be. Not only that, but PyS60 can surpass Java ME development with all the Python’s agility.

If you consider PyS60 for your commercial grade product, it all boils down to the feature set of the application, your existing team skill set and the hacker spirit. And now there exists at least one commercial vendor, besides mighty Nokia, who advertises commercial PyS60 services (hint: our company), so you won’t be alone.

We could look things in a larger perspective. In the mobile world, PyS60 is the pack leader of Python-in-your-pocket solutions. Windows CE has Python support available, but it is not officially backed up by Microsoft. This may change quickly when mobile .NET virtual machine gets DLR support making it possible to run IronPython. On Linux platforms, Python is a natural part of the platforms (OLPC, OpenMoko). Google Android and Apple iPhone will roll out their own software development platforms soon.

In the dream world, we could have a common mobile Python platform akin to Java ME of today. Java is not the only way to provide vendor neutral mobile applications: there already exists effort to get mobile cross-platform .NET implementation (Mono on embedded Linux, Red Five Labs .NET on Symbian). The future popularity of Linux phones, iPhone and Android might decide the faith of mobile Python. Could we have Python as a batteries-included feature in the future phones?

Eclipse web developer plug-in memo

Currently I work in quite wide field of software development: Python (standalone, Plone, Zope, Django), PHP, Java, Symbian and embedded Linux. I am using Eclipse for development, since it’s pretty much the only consistent platform filling my needs. The nature of work also forces me to use different computers (Mac/Windows/Linux) with different clients. This drives me to reinstall Eclipse now and then.

Below are my personal notes what plug-ins are needed to get “perfect” Eclipse set-up. Basically they are just my own notes so that I don’t need to Google everything all over again every time I reinstall. I hope the readers can find new pearls here or suggest improvements.

Eclipse setup

Eclipse has internal updater/web installer. All plug-ins are downloaded as ZIP files and extracted to Eclipse folder or installed through the internal updater. Paste Eclipse update site URLs to menu Help -> Software updates -> Find and Install, New Remote Location. You can use dummy text as the name of update site.

Eclipse WTP (Web Tools Platform)

Eclipse Web Tools Platform bundles Eclipse, Java development tools, HTML editor, CSS editor and some other generic useful stuff.

Python

PyDev is a plug-in for Python and Jython development.

Site URL: http://pydev.sourceforge.net

Eclipse update site URL: http://pydev.sourceforge.net/updates/

PDT

PDT download provides Eclipse, HTML editor, PHP editor and CSS editor.

Site URL: http://www.eclipse.org

Eclipse update site URL: http://download.eclipse.org/tools/pdt/updates/

Subclipse

Subclipse provides Subversion version control integration to Eclipse.

Eclipse update site URL: http://subclipse.tigris.org/update_1.2.x

In the installer, uncheck the integration modules checkbox or the installer will complain about missing modules.

JSEclipse

JSEclipse provides a better editor (over WTP) for Javascript files, with impressive outlining and autofill capabilities.

Download requires Adobe developer account or similar fill-in-the-fields crap.

Site URL: http://labs.adobe.com/technologies/jseclipse/

ShellEd

Syntax coloring for Unix shell scripts

Project site: http://sourceforge.net/projects/shelled

SQL Explorer

SQL editor with limited GUI capabilities. Based on Eclipse platform. Comes standalone and as Eclipse plug-in.

needs MySQL JDBC driver

Technorati tags:

Could it be possible to make viewlets simple again?

Viewlets were introduced in Plone 2.5 and they are extensively utilized in Plone 3.0. Viewlets are controlling the snippets of web pages and you must understand them if you wish to add new Plone UI elements or create new Plone themes. This new technology promises to separate Python based logic from page templates, thus making HTML authoring a cleaner process. On the other hand, it introduces new elements to Plone technology soup, further bumping already steep Plone learning curve. Could we avoid this learning bump and keeping simple changes simple.

I present not-so-rare scenario where a newbie wants to add a new piece of HTML code to every Plone page. I use a custom language bar as an example: instead of using the normal language selector, provided by Plone or LinguaPlone extension, our newbie developr wants to style a language bar suitable to his/her site theme and have it in non-standard location on the page.

Old way

In Plone 2.5, much of the Plone HTML output were controlled by two things

To add a new item to the portal header section, let’s call it a custom language bar, you had to do the following.

Voila! 7 steps. One had to understand HTML, CSS, Firebug, TAL and little bit Python now and then. This is nothing too difficult for an average Joe.

Plone 3.0 way

Plone 3.0 has a system with containers and viewlets. All viewlets are bind to a certain container, in which they can shuffled and hide (even through-the-web!) .

Pros:

Cons:

Below are steps what we need to create the imaginary custom language bar in Plone 3.0. This is not the exact recipe, but I guess you all get the point along the lines…

Phew. In this point we have at least 2 python modules, 1 page template file, 2 different XML dialects, 150 lines of Python code. You need to master following technologies: HTML, CSS, TAL, @@manage-viewlets, object-oriented Python, Zope Component Architecture, ZCML and GenericProfile XML. The five latter ones don’t actually have anything to do with the job, changing a bit of HTML code. Basically this means that anyone who wants to pimp his/her Plone must have PhD in software architectures.

Why would you need to be Enterprise Software Architecture to make a simple HTML change? I am afraid that the new Plone way of HTML authoring will drive people futher away from Plone, because they just can’t do it(tm) anymore. Plone is already enjoying bad ass learning curve. Making it even steeper and forcing people to learn things which are not really related to their jobs won’t make Plone more attractive. I am afraid that soon there might be only three people left in this world how actually have the skillset to create new Plone skins (Alexander Limi, David Convent and Denis Mishunov).

Could it be easier?

The bitter comments in viewlets tutorial indicate that not everyone loves the new way. Though we need a system like this to make Plone flexible, somewhere along the way architecture just got little too complex. Maybe we should be a bit more Grok‘ish and try to hide all this complexity from novices. These two principles help in designing a system:

My dream is that to accomplish something you need as few lines and few files as possible. Preferably one file for one thing. No changes in other files needed.

It could be something like this (indent problems below!):

myproduct/browser/viewlets/my_languagebar.pt

<html xmlns="http://www.w3.org/1999/xhtml">
<plone:viewlet="my_languagebar"  to-slot="portal-top"

default-visibility="true"
  default-order="after:logo"


bind-data=”showCurrentLanguage python:True”>


<ul>


<li>Choose Finnish</li>


<li>Choose English</li>

</ul>

</plone:viewlet>


</html>

Explanation: Let’s add viewlet my_languagebar to portal_top container and by default it should be placed after logo viewlet. The HTML code is all there. We override context variable showCurrentLanguage with inline Python script.

Now, let’s unleash the debate dogs and let them bite this blog :)

Technorati tags:

Technocrati claiming

Technorati Profile

I am adding this blog to Technocrati

Sharing RSS feeds from the feed reader

I am following pretty many good web technology blogs in my Google Reader. Compiling this RSS feed list has taken few hours and I’d like to share the fruits of my hard work. Luckily in our beautiful world of Web 2.0 this is easy.

  1. Use Google Reader Manage subscriptions -> Export function to export your RSS feed list in OPML format
  2. Upload this file to your server
  3. Install OPML Blogroll widget, by fabulous Sterling Camden, to Wordpress
  4. That’s it!

You should see the results on the right —>

The next step is to get auto export URL (via Google API keys?) to Google Reader so that I don’t have to manually sync the OPML file on my server.

I also recommend sharing your RSS feed list in Share Your OPML service.

← Previous Page — Next Page →

Copyright © Red Innovation Ltd. 2009 All Rights Reserved. | Log in | XHTML
Close
E-mail It