The Pyrite Project
Integrating Your Palm Handheld with Your Linux Desktop
Andrew Blais, Ph.D.
David Mertz, Ph.D.
Unindicted Co-Conspirators, Gnosis Software, Inc.
May 2001
The Pyrite Project has created several related tools to allow Python programmers to access and control PalmOS handheld devices. Pyrite is used to communicate with and manage the data help on Palm devices; Pyrite Publisher is used to create and distribute Doc format e-books to Palm devices. This article discusses David's experience in working with Pyrite tools, their underlying architecture, and tips for effective use of the Pyrite tools.
What Is The Pyrite Project?
We suppose our reader is like us in the following three ways. You have a brand new PalmOS handheld that promises to do all sorts of neat things. Your OS of choice is Linux. And, you want to transform documents that you can access from within your Linux environment into documents that you can access with your PalmOS handheld, and maybe even programmatically manipulate the record database on your Palm. The Pyrite Project provides this set of tools; we will focus, in this article, on the portion dubbed Pyrite Publisher that deals with document conversions.
While the Pyrite Project has some promise, and addresses some important needs, it unfortunately still has a lot of rough edges. We will try to walk you through the problems you are likely to encounter.
What You Will Need
Pyrite Publisher is written in Python, and depends on some code provided by yet another pair of packages called Pilot-Link and Pyrite. Consequently, you will need the following five things.
1. Python 1.5.2, or greater. 2. The development headers for Python. 3. A C compiler -- gcc worked for us. 4. The pilot-link library, 0.8.11 or greater. 5. The header files from pilot-link. 6. The development headers from pyrite |
We will assume that you don't need help with (1), (2) or (3). There are two ways to get (4) and (5). Both methods will get you the needed header files and libraries.
The first method employs RPM. Gotohttp://www.rpmfind.net, and download the RPMs for pilot-link and pilot-link-devel. If you're using RedHat 7.0, these two links should take you to precisely what you need. The first will get you the headers, and the second will get you the libraries. See the Resources section below for links to these RPMs.
The second method is for those of you who don't have access to, or don't want to use RPM. Simply, download pilot-link, and install by hand. Download the source (see Resources). Use the standard procedures to install, namely:
bash session for installing pilot-link source
$ tar zxvf pilot-link.0.9.3.tar.gz $ cd pilot-link.0.9.3 $ ./configure $ make $ make intsall |
To get the development headers from Pyrite first download the source (see Resources). Next, do the following:
bash session for installing Pyrite source
$ tar zxvf pyrite-0.9.3.tar.gz $ cd pyrite-0.9.3 $ ./configure $ make |
One of the Makefile's has an error, so use your favorite
editor to open 'i18n/Makefile. On line 26, change:
Required fix for Makefile error
# mkdir -p $(DESTDIR)$(LOCALEDIR) ; mkdir -p $(DESTDIR)$(LOCALEDIR) ; \ |
Otherwise, you will get the following error message:
make[1]: Entering directory `/root/PAPERS/palm-stuff/pyrite-0.9.3/i18n' if [ "yes" = "yes" ]; then \ mkdir -p /usr/share/locale ; /bin/sh: -c: line 2: syntax error: unexpected end of file make[1]: *** [install] Error 2 make[1]: Leaving directory `/root/PAPERS/palm-stuff/pyrite-0.9.3/i18n' make: *** [install] Error 2 |
Finally, run make install.
You now have all the header files and libraries that you need to install Pyrite Publisher.
We encountered several problems installing the Pyrite libraries
and headers on our Mandrake 7.1 system. These issues are not
directly Pyrite's fault, but you might find you have to install
a few dependencies on your system. First, configure
complained that it couldn't find the python config makefile.
We downloaded a suitable python-dev RPM, which we downloaded
and installed with no trouble (see Resource).
Next, configure complained that it couldn't find the
pilot-link library. We knew that pilot-link was installed, and
that we had /usr/lib/libpisock.so.3 and
/usr/lib/libpisock.so.3.0.1. But, we didn't have a generic
libpisock. This was resolved by creating the following link
in /usr/lib:
Creating an non-numbered libpisock.so link
$ ln -s libpisock.so.3.0.1 libpisock.so |
Subseqently, configure complained that it couldn't find the
pilot-link library. So, we searched rpmfind for the
appropriate pilot-link-devel-*.rpm. However, we encountered
the infamous "only packages with major numbers <= 3 are
supported by this version of RPM". So, we used the force:
we got the source (see Resource). Then, we did the following:
bash session for installing pilot-link headers
$ tar zxvf pilot-link-0.9.3.tar.gz $ pilot-link/include $ cp * /usr/local/include |
Finally, configure was happy. And make ran fine. We
thought that we were on our way, but make install died
because of the already mentioned typographical error in
pyrite-0.9.3/i18n/Makefile. We made the change described
above, and even make install worked.
On our Slackware 7.1 system, python-dev was installed by
default, but pilot-link was not. Pilot-link needed to be
configured with the --without-cpp option. Our glibc and gcc
had been upgraded from the 7.1 default, which may or may not
have led to this option being required.
Aside On Pyrite
Pyrite-0.9.3 provides a library that can be used to write Python code that will communicate with a PalmOS device. If you are a Palm user who has made it to this document, you will more likely than not also be a user who will eventually want to write such code. If so, you will be in luck, because you will have the resources to do it.
One valuable source of information is to be found in
pyrite-0.9.3/doc/prg/html/book1.html, which is the root of
the Pyrite Programmers Guide. Pyrite depends on another
library called Sulfur, and there is helpful documentation to be
found in pyrite-0.9.3/Sulfur/doc/html/book1.html, which is
the root of the Sulfur Programmers Guide. The relation between
Pyrite and Sulfur can be better understood by noting that as
Rob Tillotson developed Pyrite, he found that he had also been
developing code that was useful outside the context of palm
applications. This code became Sulfur.
In or under pyrite-0.9.3, you will find the following five Python modules with their various classes.
1. Blocks.py:
Block, Record, AppBlock, CategoryAppBlock, SortBlock,
Resource, and PrefBlock
2. Database.py:
Database, Slice, and CategoryIterator
3. Store/Store.py:
BaseStore and Store
4. Application/Application.py:
Application, PyriteState, and PyriteCLIAppContext
5. Conduit/Conduit.py:
Conduit
Blocks and Database are imported along with Pyrite, but Store, Application and Conduit are not., There are also additional modules, classes and methods, but the listed ones received the vast majority of the attention in the documentation. In this context, we will only summarize these modules and some of their methods.
Block objects represent and transform data to and from PalmOS and Python readable states. For example, Python stores the time in a tuple, and a palm stores the time in the form of two packed bytes. Suppose we want the time 10 hours, 30 minutes, and 0 seconds. Then,
Python code for creating a Palm date
from Pyrite import Blocks time_tuple = (10, 30, 0) palm_time = Blocks.tuple_to_palm_date(time_tuple) |
will put 4032 in palm_time, which is the way that a Palm
internally represents the time.
A Database object is an accessable database with an .append()
method, a .new_record() method, a .delete() method, and a
.next_record() method. This is all quite interesting stuff,
but it also goes quite beyond our aim here. We refer our
readers to the above cited resources.
Installing Pyrite Publisher
Of course, begin by downloading the source (see Resource). Then, do the following:
bash session for installing Pyrite Publisher
$ tar zxvf pyrite-publisher-1.99.0.tar.gz $ cd pyrite-publisher-1.99.0 $ python setup.py install |
One would hope that this is all that you needed to do. Unfortunately, this is not true. There are several more poblems with Pyrite Publisher. Here is how we resolved them.
Some More Fixes
Again, Pyrite Publisher promises to transform documents into files that can be uploaded to and read on a PalmOS handheld. Unfortunately, Pyrite Publisher doesn't do what it is supposed to do right out of the box. We found that we needed to make several fixes before we could get it working. For someone with no knowledge of Python, this is really a big problem. The following code changes will allow you to fix Pyrite Publisher just enough to transform a document such as an html document into a form that can be uploaded to a PalmOS handheld, and then read them conveniently on a train or in a cafe.
Several changes are needed to PyritePublisher/plugin_HTML.py. In
Plugin.eof(), on line 79, change:
Required fix for plugin_HTML.py (1)
# self.fmt.add_flowing_data(p.anchorlist[x]) self.fmt.add_flowing_data(self.parser.anchorlist[x]) |
On line 99 (same class), change:
Required fix for plugin_HTML.py (2)
# DocToolkit.entitydefs import entitydefs PyritePublisher.entitydefs import entitydefs |
The next fix is to PyritePublisher/plugin_TealDoc.py. On
line 32, change:
Required fix for plugin_TealDoc.py
# DocToolkit import metrics PyritePublisher import metrics |
Apparently, these changes were necessary since the authors of this project failed to remember that they had changed the name of their project big time ouch.
The next fix is to PyritePublisher/doc_database.py. In
DocWriteStream.close(), on line 135, change:
Required fix for doc_database.py
# db.setRecord(0x40, uid, 0, bookmark_pack(t,p)) db.setRecord(0x40, uid, 0, 'THIS IS A TEST STRING') |
This last fix is undoubtably a really ugly hack, but our goal
here is not to beutify code. Our fix has the strange side
effect of inserting odd looking bookmarks into our pdb files,
which appear as a series of y's with umlauts. Apparently, the
function bookmark_pack() is intended to create Palm-style
bookmarks, but the function is missing, and we are not sure of
the exact format. With this hack you can transform documents
on your Linux machine into documents that you can read on your
PalmOS handheld. Hopefully the authors will create or restore
the needed function.
How To Use Pyrite Publisher
Using Pyrite Publisher is relatively simple. In
pyrite-publisher-1.99.0/PyritePublisher, you will find
pyrpub. Supposing that the document that you want to
transform into PalmOS readable form is file.html, the command
is simply:
Converting an HTML file to PalmDoc
$ ./pyrpub file.html |
You will find a file named file.pdb, which can be read by a Palm.
The next step is how to get file.pdb on your PalmOS handheld.
You can use a nifty program that comes with pilot-link called
pilot-xfer. Assuming that you have connected your Palm to
/dev/cua1, issue the following command:
Transfering the PalmDoc file to the Palm
$ pilot-xfer /dev/cua1 -i file.pdb |
Then, press your hot-button on your craddle. Actually, you will be
instructed to do this on the command line. Then, you will find
file.pdb on your Palm.
A limitation of Pyrite Publisher is that it doesn't directly
convert PDF or Postscipt files to pdb files. Luckily, there is
a simple workaround for this. The utility pstotext
can transform a Postscipt file into a text file. To
generate a pdb file, first transform the ps file into a text
file, and then use Pyrite Publisher to convert the text file
into a pdb file. If the file that you want to transform into a
pdb file is a PDF file, first use pdf2pdf to transform the PDF
file into a postscript file, and then proceded as describe
above. Unfortunately, this isn't a foolproof method, since
pstotext and pdf2ps aren't entirely reliable, but when they
work, you be able to generate pdb files.
Other Things
To find packages related to the Pyrite libraries and Pyrite Publisher, try searching Freshmeat. We think that you might want to look at some of the following packages.
* There is KVM, the K Vitural machine, and the Connected Limited Device Configuration, CLDC. This is a package that lets you write palm applications that can be run on X11.
* Kpilot makes it possible for your palm to exchange data with your Linux box. Kpilot uses plugins to exchange data between your Palm and other applications like KOrganizer or POP3/SMTP mail servers. Using Kpilot, you can read and edit documents install new programs on your palm.
* Jan Schaumann has done some interesting things with the palm within the context of the Linux OS.
* Plucker also enables the Linux user to read HTML documents offline. Plucker claims to support clickable images, italics, multiple databases, configurable display parameters and stylus options, compression, and Python conduits and parsers.
* DataViz' Documents To Go is a sophisticated document conversion package, but only for Win32 and MacOS. If you happen to use these platforms, you can convert document formats such as MS-Word, MS-Excel, Word-Perfect, PDF, and others. You can even edit documents in many of these formats on your Palm. Of course, you have to pay $50 for it.
For links to information about these packages, see Resources below.
Summary
We like the concept behind the Pyrite Project, but we are sorry that the truth is that it doesn't quite live up to its promises. Pyrite Publisher sort of works, but only if you know enough Python and system administration to fix it, and even after it is fixed to the point of doing some work, it still has an overly large collection of weird bugs. There is also the problem that Pyrite Publisher doesn't directly convert Postscript and PDF files. We are not totally negative, however, since it clearly seems to be on the cusp of fulfilling its aim. We hope that Rob Tillotson and company will continue with the Pyrite Project. Linux-using Palm owners can certainly use it.
Resources
The RPM for the libraries of pilot-link can be found at:
ftp://rpmfind.net/linux/redhat/7.0/en/os/i386/RedHat/RPMS/pilot-link-0.9.3-10.i386.rpm
The RPM for the header files of pilot-link can be found at:
ftp://rpmfind.net/linux/redhat/7.0/en/os/i386/RedHat/RPMS/pilot-link-devel-0.9.3-10.i386.rpm
The source for pilot-link can be downloaded from
http://prdownloads.sourceforge.net/pilot-link/pilot-link.0.9.3.tar.gz
The source for the pyrite libraries and header files can be retrieved from here:
http://www.pyrite.org/dist/pyrite-0.9.3.tar.gz
To download a suitable python-devel RPM, search at:
http://rpmfind.net.
The source for Pyrite Publisher can be downloaded from:
http://www.pyrite.org/dist/pyrite-publisher-1.99.0.tar.gz
There is a RPM of some tools that are similar to those found in Pyrite Publisher. See:
ftp://ftp.rpmfind.net/linux/contrib/noarch/noarch/palm-doctoolkit-1.1.4-1.noarch.rpm
The KVM package can be found at:
http://www.extreme-java.de/kvm/kvm.html
Information about Kpilot can be found at
http://www.slac.com/pilone/kpilot_home/
Jan Schaumann's Palm/Linux site:
http://www.netmeister.org/palm/
Plucker can be found at
http://plucker.gnu-designs.com/
DataViz Documents To Go information can be found at:
http://www.dataviz.com/
About The Authors
Andrew Blais divides his time between homeschooling his son, writing for Gnosis, and teaching philosophy and religion at Anna Maria College. He can be reached at [email protected].
David Mertz worries that if the density of transistors doubles
every eighteen months, then he may lose his next computer in a
haystack. David may be reached at [email protected]; his life pored
over at http://gnosis.cx/publish/. Suggestions and
recommendations on this, past, or future, columns are welcomed.