Logging with LogMan

This is my first post on our company blog and I thought I’d tell you something about LogMan, which is developer’s utility for getting logging messages from Symbian device over a USB cable. It is written by me and mostly on my own time. I started the project because I had to do a Symbian excercise for university course and I thought I’d do something useful instead of quickly tinkering something small and easy.

LogMan supports both C++ and Python. With LogMan, you can send data to the same location from Python and C++ in real-time instead of using log file(s). Browsing through multiple log files can be tedious and you can’t see the debug output while using your application, because on Symbian you must read the file after the debugging session - Symbian cannot share opened files between applications. It is also possible that you create too much log and you run out of Phone internal memory. LogMan helps by removing the use of log files and you never run out of disk space because messages can be stored directly to PC. On simulator, the messages are also sent to RDebug (%TEMP%\epocwind.out). Surprisingly I have not seen a Python module, which would enable use of RDebug. Even though it is very easy to implement. With RDebug, there is no need to open a serial port on simulator for reading the logging messages.

Of course, I tried to use RDebug on device first, but I never got it working. I also tried REcmt, which is supported on S60 only and the service just kept on crashing on my phone. This is why I decided to write LogMan. Plus both are closed software, which effectively prevented me from fixing the problems.

Just wondering what kind of benefit Symbian or Nokia gets from keeping development tools such as these closed? What is there so secret about them? It didn’t take me very long to write the first working version. *sigh*

The use of LogMan is similar to RDebug. There are static class methods, which are a bit slower but easier, and instance methods. Check the project’s homepage for more examples.


#include "LogMan.h" //RLogMan
RLogMan::Log( _L("Hello world ") );

I recently added a new feature for LogMan, which allows you to log stack and heap usage of the current thread with one function. When you are unsure about your heap or stack usage, these might come handy. Of course there is some memory used when calling these so take that into consideration. Python can access MemoryInfo only, which logs both stack and heap (Well, I got a bit lazy at that point). There are equivalent macros for these, so check them out from “logmanutils.h”.


// Store this as a member of your class, for example
RLogMan logman;

logman.HeapInfo();
logman.StackInfo();
logman.MemoryInfo(); // Both stack and heap
logman.Close();

The output from MemoryInfo is something like this:

StackInfo
Free:1039000, Used:9576, Size:1048576
HeapInfo
Free:25856, Used:101004, Size:126860

Browser for PyS60

I have been trying, unsuccessfully, to get Browser Control working on PyS60. In a nutshell, CBrCtlInterface wrapper for Python. I have developed it against PyS60Community version in Launchpad. See /src/appui/appuifw/. I have used LogMan extensively to debug the extension so if you want a real example, check out “browsercontrol.cpp”.

Browser Control would allow one to embed a browser into his PyS60 application, which would be quite cool. No need to do user interfaces with “appuifw”, which is not very portable. With Browser Control, one could create his user interface with html and javascript, which are a lot more portable indeed. Less work leads to more time. And what is time?… it’s money. Or so I have heard. And being able to handle events with Python instead of C++ is another bonus.

Unfortunately, the API is not very stable as you can see by searching for “CBrCtlInterface” at Forum Nokia. The browser worked fine on the simulator with small pages such as “www.google.com”, but it crashed miserably with larger pages. The crash happens in browsercontrol.dll when calling e32.Ao_sleep() in Python. On device it was unable to open any page and crashed instantly when trying to load one. With LogMan, I was able to verify that the crash happened in e32.Ao_sleep() on device also. This reminds me to test on device all the time, which I didn’t do for the first versions. But this is why I added the memory logging feature to LogMan, but it only revealed that I was not out of stack or heap. At least not before the browser started to mess around.

I tried to compile the WebKit myself to see what is going on, but the build instructions didn’t work and the build scripts are written in Perl(my eyes started to hurt). What a mess. I don’t wonder anymore if there are bugs in browsercontrol.dll. I finally gave up because my idea pool dried up. Any help getting the wrapper working would be very much appreciated.

Plans for LogMan

I’m planning to add remote shell interface so that you could control your phone from PC. I want access to the file system first. Transfer files, list folders and such. If you have TCB rights ( or hacked phone ) you can speed up development remarkably by simply replacing your binaries in \sys\bin or your Python files with new versions. No need to install sis files and fiddle with certificates and play with memory cards. It would be so nice… I can use 1 day in a week for a personal project so this may happen in near future ;)

PyS60 application release build toolchain

A common question for Python for Series 60 newcomers is how to build standalone Symbian applications from Python source code. We have been using Makefile based toolchain internally. I describe it in this picture, I didn’t bother to add thumbnail for the image, since it’s a 3400 pixels wide diagram.

The diagram describes building a PyS60 application with some Python extensions (Symbian native C++) mixed in and bundling it all to one downloadable SIS file. The application will appear as any first class S60 application in the menu and the user does not know it’s running Python internally, besides bad installation experience (it challenges Microsoft installers with all those unnecessary yes/no questions), extra uninstaller entries and slow start-up time.

The biggest problems are caused by embedded SISs (SIS inside other SIS files) which are not treaded very wel by several Symbian parties.  In theory, it could be build one monolithic SIS, but you’d need to recompile PyS60 from scratch and patch UIDs inside it for your own UIDs received from symbiansigned.com. We are planning to explore SCons based build solution to address this problem, since Makefiles are a bit unflexible with tasks like PKG file and UID range generation.

Here is a PKG file example for final user distributable SIS file.

Also, see UIKludges project for additional details for PKG files of Python extensions.

You need to have

You need to master

Pros

Cons

Ps. I would have put this thing to wiki.opensource.nokia.com, but their webmaster email address is non-functional and one cannot upload images to their Wiki.

Symbian digging its own grave with symbiansigned.com

Edit: Made correction to iPhone SDK price/added self-signed option

I have been recently involved in Symbian development. Unfortunately, I could say. Symbian has made the life of Symbian developers headacheful. The main problem is that the application signing is made unnecessary cumbersome. Getting a required developer certificate to just to run your one liner test code is a long road paved with curses, stress and ponder how a world class company can screw up things so badly.

I write this post to vent out the frustration. Hopefully someone picks up the feedback presented here. Personally, I don’t care anymore really. After explaining to the customer once “I am afraid that we cannot have the demo in this meeting, since we were unable to get the developer certificates in time. It was nice you did summon all of your executes, though.” my sympathy towards Symbian has fallen to zero.

In the long run, I believe, this will hurt Symbian when developers leave the platform. When this happens Symbian will miss 1) developer base 2) the wow value of all new cool things when hackers move to happier platforms - after all you don’t want to work with things causing a headache even if you are paid generously. Professional Symbian capable developers are already rare and scaring away the potential candidates (students, open source developers, freeware developers) won’t really help the matter. Of course, for us, who are already in the business, it’s good since we can raise our hourly fee and die as rich dinosaurs (remember COBOL).

Looks like the root problem is the cocky attitude: “We are the biggest mobile operating system in the world. You are not our customer, the biggest mobile phone brands are. Thus, you are an insignicant fly and we don’t need to listen to you. Please go away.” The big boys tell that this very same reason lead to Motorola’s fall from the number one mobile phone vendor position - so let’s see if the history is repeating itself.

The problems of Symbian development process, especially signing, are well known. Symbian Signed forum category has 120 000 views. Symbian signed support requests has 57 000 views and 2000 messages, since November 2007. Over a thousand support requests in four months. Symbian has money. It could fix the problem if it has will. If there is will in the organization and the work just does not progress, the corporate world has a simple solution: fire the management and hire capable responsible persons.

I won’t go the details of the problems in this post. I have prepared a post containing over 50 bulletins with detailed steps to get the developer certification process done and it lists every little mistake in Symbian Signed process. Unsurprisingly, the post mentions fuck and hell over twenty times, so I am reluctant to post it on our company site. So I sketched a manager comprehensible overview of the state of the process in this post. To justify my criticism, I compare Symbian to Apple iPhone and make some suggestion how to fix things.

Certificate costs

Symbian: 200$/1.5 years (you need a publisher id to get a developer certificate) + 20$ per signing round

Update: You can still make self-signing certificates yourself with limited capabilities (no GPS), but symbiansigned.com doesn’t bother to mention it.

Apple: 99$/year (for testing and distribution applications)

Suggestion: Make the developer certificate free. It used to be free, but based on this post, Symbian signed claims that they do not have enough CPU power, because of all kind of nasties are knocking their server. Come on! it’s Internet. You are hosted by Cidercone and by knowing how much they charge for the hosting I believe you can get every possible server power in the world to generate your keys.

This development start up cost automatically excludes third world people, freeware makers and students i.e. the future Symbian professionals. It’s hard to maintain a platform if there is not enough skilled people to work with it. There won’t be meat into Nokia’s and Sony-Ericsson’s innovation machines.

Development start lead time

How long it takes to get “Hello World” code to the phone.

Symbian: 8 workdays

Apple: 6 hours

Complete Guide to Symbian Signed guide (manual) is 20 pages PDF. How often you need a user manual to use a web site? Even with this page amount, the manual lacks crucial pieces of information which you need to hunt and ask help from the Symbian Signed support forums. I had to make 3 support request for Symbian Signed and 1 for TrustCenter.

Set-up process

Symbian: Create 3 user accounts (symbiansigned.com, developers.symbian.com, TrustCenter). Fax your passport and some signed papers to TrustCenter Germany. Wait when your request is manually processed. Ask TrustCenter what I do with the link they send me (hint: the only working web browser for their site is the very same browser you orignally submitted your application from). Convert TrustCenter key to Symbian specific format using obscure BAT tool. Use half-baked Windows EXE to generate files. Ask at the support forums permission to be allowed to create the Publisher id.

Apple: Download SDK. Click “I Agree.”

Symbiansigned management team must be either incomplete or inexistent, or they just hate their users. Looks like no one didn’t bother model or test this process, since the flaws are so apparent. Why do we need three user accounts? Why do we need a user account at all, isn’t my credit card payment enough? Why there can’t be a single web page on symbiansigned.com where I could do these all things without? Also, Symbian could do these things without third parties (TrustCenter) who are badly integrated to the process.

Web site and information reachability

Symbian: Two sites with different user accounts. Information is spread around and hidden behind various for registred users only pages. Often Symbian PDFs (yes they seem to prefer zipped PDFs) and web pages cross-link to other sites and these links are broken. Their internal site search engine does not give meaningful results. Google cannot search pages, since they are hidden behind the registration. Web forms don’t work. Navigation paths are unclear and you often need to guess where a certain piece of information could be.

Apple: One site - and it just works.

Why Symbian hides a crucial BAT tool in the page labyrinth of developers.symbian.com and behind login? Does the 100 lines of BAT code contains a very hidden trade secret? I doubt. (I also doubt that a BAT tool is a good way to solve a problem which could be fixed with a web form or Javascript, but that’s another matter.)

I think this is the part of the attidude problem. SymbianSigned.com mission is not to “secure” the development and control the developers of the platform. It’s mission should be easily enable developers to start development and release their software for Symbian. The meaning of “easily” includes that the web site is working and up to the standards. Please hire a usability expert (we are here) and get it fixed. This problem is not solvable by writing yet-another user manual PDF.

Giving feedback

Symbian: Support like goes to the Symbian developer forum. A bulleting board with unorganized, angry, posts. They don’t give a feedback form to report the problems on Symbian Signed.

Apple: “There are three great ways to get answers to your questions about ADC membership, products, and services: visit our Frequently Asked Questions web pages, submit your questions or feedback using the form below, or call us to speak directly to a worldwide support agent. We look forward to assisting you.”

I’d help symbiansigned.com to get its site better if I could. When there is a broken link, I can hit feedback and ask them to fix it. But looks like they want to be ignorant about the matter - if I report a problem on their forum I bet no one will ever fix it. There won’t be even “thank you.” Probably no one ever reads my post. On the other hand, when having such non-functional service, this feedback channel would be flooded in overnight.

Conclusion

Symbian has unforgiveable screwed up with its signing process and doesn’t seem to get the feedback from the developers who suffer from it. Steve Ballmer’s famous quote “developers, developers, developers” refer to what’s the important factor making the platform succesfully in long term. This is especially crude when you know that things used to be good.

Copyright © Red Innovation Ltd. 2008 All Rights Reserved. | Log in | XHTML
Close
E-mail It