Live from Tokyo: Tech Blog

Python SOAP problem

I just installed SOAPpy097 and when I run SOAPtest.py I get a exception: xml.sax._exceptions.SAXParseException: :5:44: not well-formed (invalid token)

What's really surprising is that a simple search on Google for it gave me no clues! I'm just hoping that someone will hit page this and give me a clue.

Come to think of it, recently my biggest banes in life recently has been from tools that use XML parsers (i.e Ant & SOAP). The tools are fine but XML parsers are so terse in their error messages. It's as if they are offended that you have presented them with something dirty. It reminds me of the time I gave the girl at the Tower Records checkout an HMV point card...

The full trace is:Traceback (most recent call last):
File "C:\bin\Python\SOAPpy097\SOAPtest.py", line 717, in ?
z = parseSOAPRPC(y)
File "C:\bin\Python\SOAPpy097\SOAP.py", line 2804, in parseSOAPRPC
t = _parseSOAP(xml_str, rules = rules)
File "C:\bin\Python\SOAPpy097\SOAP.py", line 2788, in _parseSOAP
parser.parse(inpsrc)
File "C:\bin\Python\Python22\Lib\site-packages\_xmlplus\sax\expatreader.py", l
ine 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "C:\bin\Python\Python22\Lib\site-packages\_xmlplus\sax\xmlreader.py", lin
e 123, in parse
self.feed(buffer)
File "C:\bin\Python\Python22\Lib\site-packages\_xmlplus\sax\expatreader.py", l
ine 211, in feed
self._err_handler.fatalError(exc)
File "C:\bin\Python\Python22\Lib\site-packages\_xmlplus\sax\handler.py", line
38, in fatalError
raise exception
xml.sax._exceptions.SAXParseException: :5:44: not well-formed (invalid
token)

Logging and Exceptions

I'm not sure if this is going to be a rant or a comment. It seems to me that the code that logs configuration errors is the least tested code. One reason for this is because during development developers tend to set up their systems correctly so configuration errors happen rarely occur.

Later when someone comes to set up the system again to maintain or use it, they inevitably make mistakes and trigger exceptions which causes messages to be logged, often for the first time. The usually code does it job and something is logged but in practice the error message only tells you what occurred but gives no clue on how to fix it. Often the reason for the exception is not preserved when exceptions are not chained.

Right now I have problem but I'm refactoring code, deliberately not fixing my environment until the logs are printing something useful. Through deep debugging I already know what the cause is but I shouldn't need to debug to know that a config path is wrong. The code is improving.

PhotoBlogging

Just as Justin's essay on Moblogs was published I managed to finish a Python script to post emails containing attached photo's from my Au mobile phone to my weblog hosted at LiveJournal. I will be working on a connector for the Blogger API next to be able to post to Moveable Type and other blogs soon.

Format your code

When we started work on Mojo we pretty quickly developed a consistent coding style and made new developers stick to. As a result the code is pretty much formatted identically everywhere. It's a good thing to get this issue sorted out at the initial interview time. If someone won't format their code to a pre-agreed standard then they will probably not do a lot of other things that are required in order to have an orderly software development process.

The selection of which style to use is a religious debate and thus should be decided by the wise benevolent dictator. Another good policy is not to allow debate on programming style during work hours. Outside work hours you can code however you like but don't waste time whining about how much you don't like the decided upon style while we are trying to get some work done.

I'm convinced that inconsistent formatting and debates about which format to use are much bigger time waster than the gain or lack of productivity between one style or another.

PopFile

I am checking out Popfile a local POP Proxy that marks spam with a special header based on Bayesian statistical analysis of the SPAM contents.

Initially Popmail knows nothing about SPAM. You can make any number of buckets to sort mail into and then you train the filter as to what is spam and what is not. If it mis-classifies something you can go back and retrain it on that particular mail. It is interesting to see which words that it thinks are spam indicators.

I'm just waiting for more mail to arrive in my mail box.

How to Be a Leader in Your Field

This paper was written for students but looks interesting to me for anyone who is trying to get noticed in the their field. How to Be a Leader in Your Field.

Spambot Beware

Spambot Beware is a nice collection of anti-spambot tools that I found on this list of anti spam tools.

Build Process as History

Today I was maintaining some code that one of my co-workers wrote over a year ago. Unfortunately he is longer here. Everything went fine until I had to recompile an RMI generated class.

When I compared the previous class to the one generated from my code it was very different and I couldn't explain the differences. Aftere tinkering with the options I turned off 1.1 compatibility and the differences became understandable. Unfortunately this generated file had been checked into CVS but the method by which it was generated was not in the build process. Thus we had no memory of what options to use should we have to rebuild it again.

The moral is that the build script as well as making your life easier is also a document describing how to build the project from scratch.

In my case today fixing the option was trivial but if something more complicated had been checked in I would have been in trouble.

Design Patterns in Java

I hate the examples in the Design Patterns book. I hope that the examples in this book are better: The Design Patterns Java Companion.

Publishing-Oriented Templater

Mark Paschal's Python based Publishing-Oriented Templater is an engine for publishing HTML files with embedded Python code by FTP.

I usually shy away from systems where the code is mixed in with the templates but I am intrigued by this software. Too bad I'm too busy to try it out! I think the Radio Userland's upstreaming architecture is really clever so it is interesting to see a similar approach done in Python.

I have a idea brewing at the moment and this software is similar to it...

Documenting Ant files

Good software engineering process implies that code should be thoughtfully designed and thouroghly documented but often the build process is cobbled together over a long period of time and a complete mystery to every one except the author.

What I need now is a good book/article on "Refactoring and Documenting Ant Builds".

MailToRSS

Decaf writes, "MailToRSS will merge my incoming email stream with my news stream. Produce RSS from mailbox indexes, provide links to read mail items, provide forms with which to reply to email ala weblog comments."
MailToRSS

Complexity

I'm just checking some Oracle table creation SQL that my installer program has generated against the SQL that was given to me. Once the installer is done I'll be able to generate the Postgres and Sybase versions automatically.

On looking I can see that my version in several cases is writes VARCHAR2 whereas the given version has CHARACTER VARYING.

Shelley Doll's article on SQL basics: String data types says:
"The CHARACTER VARYING, CHAR VARYING, and VARCHAR keywords are synonymous." but I can't find the exact same information in the Oracle Reference.

This is typical of the accidental complexity that arises dealing with Oracle.

I can empathize with the reviewer of Oracle8: The Complete Reference who writes: "This book is huge, at 1344 pages it is huge. But not at all useful. Information is too darn difficult to find, and more often that not it is simply not there."

Clunky Camera

Jon Robb has a picture of European phone camera in his blog. I didn't realize that they were so clunky!! Here's some pictures of my phone taken in a mirror.

This picture shows the front display which allows see what you are taking when you point the camera at yourself.

A more readable JavaDoc

I love (well written) JavaDoc. It's the best thing since Literate Programming but the biggest pain is that to do any formatting you have write HTML in your Java comments which makes them much less readable at the source code level.

Writing the comments in reStructuredText looks much better as you can actually read them. Well that was the goal of the project.

"reStructuredText is an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and parser system. It is useful for in-line program documentation (such as Python docstrings), for quickly creating simple web pages, and for standalone documents. reStructuredText is designed for extensibility for specific application domains."

I hope someone with time on their hands hacks the JavaDoc-er to accept reStructuredText as well.

For examples see the ReStructuredText Quick Start and it's source.

Providers of bad RSS feeds should be shamed into fixing them.

"...And of course, most popular aggregators don't care, today." I see this as analogous to browsers accepting almost any old garbage as HTML. When a browser parses the HTML it knows that there are errors but ignores them. It is because of this that 99% of pages that haven't explicitly been through a validator are broken in some way.

In the same way, aggregators know when a feed is bad but don't complain. If the aggregators marked the bad feeds with an un-smiley face which linked to a validation service then the people who made the bad feeds might be shamed into fixing them as everyone in the communinity can see that it is broken.

Originally posted as a comment regarding Computerworld's RSS feeds that don't validate

My name is Laura Betterly and I am a SPAMer

Laura Betterly's sends SPAM for a living and she figures her income will be $200,000 this year.

Logo the next generation

We were just joking about programming in Logo but Logo has come along way since I last looked at it

StarLogo is a specialized version of the Logo programming language. With traditional versions of Logo, you can create drawings and animations by giving commands to graphic "turtles" on the computer screen. StarLogo extends this idea by allowing you to control thousands of graphic turtles in parallel. In addition, StarLogo makes the turtles' world computationally active: you can write programs for thousands of "patches" that make up the turtles' environment. Turtles and patches can interact with one another -- for example, you can program the turtles to "sniff" around the world, and change their behaviors based on what they sense in the patches below. StarLogo is particularly well-suited for Artificial Life projects.

DOS & I don't MS...

"Nothing more than the whim of a 13-year old hacker is required to knock any user, site, or server right off the Internet.", Steve Gibson writing about several Distributed Reflection Denial of Service Attacks that brought down his site.

This is must read, if you are interested in Security and especially if you have a Windows machine on the net all the time.

Who's pointing your site

An article on mining refer data Link Feedback - see and show where your visitors come from!

Installing Apache

I just downloaded and installed Apache from the source on my Linux machine. Hey! I know that's not a big deal but there's a first time for everything!

Use the Source!

Where do people usually keep the source packages they download?

Later. On our machines we download the tar file to /tmp and the untar into the net administrators home directory. Once it is build we let root take over and install it.