Patrick Plaggenborg

Object history

People do not realize they are leaving a growing amount of tracks because of digitalisation. Phone calls are being stored and internet traffic is being logged. People are not just blind for the digital history these digital tracks form, they are also blind for history in the analogue world.

Even without RFID, objects have their own history. They all lead their own life. Some are used intensively and might build very strong relationships with people. Others are destined to a lonesome stay on shopping shelves. Objects collect human emotion and experiences. This emotional history is invisible to strangers.

While browsing a store with second-hand goods I found a very old plastic military toy. The turret was broken off and it only had 1 wheel. My first thoughts were, how can they charge € 0,25 for a tank like this? Would anyone ever buy such a worn toy? But then I thought about all the things the object might have been gone through.

My project’s goal is to let people look at objects differently, by making the unforeseen emotional history visible. In this project users are able to digitally construct that emotional history by telling their own audiovisual stories about the objects. An object looking worthless at first sight will be appreciated when listening to stories about others’ experiences. Hidden emotions are revealed.

With a future pervasive RFID infrastructure and mobile devices capable of interacting with the ‘digital’ objects, this hidden emotional history can be revealed. The digital body RFID is creating, can build a collective emotional memory, making it possible for objects to carry the story of their own history. This is the main point of the supply chain application of RFID.

History parallel

A mobile device is used to view emotional stories about the object. A critical stance on the supply chain history collection is taken, and a parallel is created between the descriptive history of EPCglobal or Thinglink data and the emotional history of object stories: digital history and analogue history. The user can go for the descriptive history and retrieve manufacture, transport, retail or usage information. On the other side of the screen the parallel of emotional history is displayed. A list of currently added stories is displayed here. The user can then select a story and listen to it. In the end the goal is to be able to expand the collection of stories about an object by letting the users of the system record their own.

Finally every little part of the application is working together:

1. Every object has its own unique identificating code, stored in an RFID tag.
2. The SocketScan software on the PDA is reading the tag with the CompactFlash RFID reader and is sending virtual keystrokes to Flash
3. Flash is recognising the unique ID and is retrieving the object’s information from the webserver.
4. Object information is stored in a MySQL database.
5. Flash likes to communicate in XML so Flash is accessing a PHP script which is forming XML from the SQL data.
6. The external website running the PHP script is also hosting the object photograph that is being displayed next to the Descriptive History.
7. The amount of stories and their unique names are also found in the XML data.
8. Thumbnails of the stories are stored als jpg images on the webserver.
9. The Open Source Red5 Flash Communication Server is hosting the actual stories in the Flash Video format (flv).
10. Flash is streaming the flv stories from the Red5 server.

For the graduation exposition I’m working on the ability for visitors to add new stories to the objects available.

After waiting two weeks for a reader that was supposed to be shipped within 2 days, yesterday I got the message that it would take another 5 days. With my exam this monday that is of course way too late. It was quite annoying to get to hear this on such a short notice. Thankfully I was able to get hold of the last reader in stock at the Dutch distributor of Socket products.

In the mean time I’ve been working on the keylogging class for Flash. After installing the socket reader it was quite easy to read and write information to the RFID tags with the Socket Demo application. Unfortunately my keylogging class did not work right away. The onKeyDown handler does not work with the SocketScan keyboard wedge software. Flash did not notice anything. The onKeyUp handler does work, but the keycodes that are send do not resemble an ordinary keyboard at all!

Using a hardware button to activate the scan worked fine and SocketScan did send the text to Flash, but Flash was unable to recognise the text. All that was recognised were keycodes 0 and 115. The keystrokes did however turn into actual text when sending it to a Flash TextField. It was necessary to rewrite a keylogging class to check an invisible textfield for a tag ID.

Earlier I mentioned the CompactFlash RFID Reader Card by Socket Communications. After the Mobile Bristol Toolkit this device was my second choice. It’s shipping with a demo application to read and write tags and SocketScan software sending virtual keystrokes (called wedge software) to any active application on the Pocket PC.

The Mobile Bristol toolkit was my first choice because of the nice integration with reader and the application. Sending virtual keystrokes is not such a tidy solution, but it does the trick.

Before rushing out to order the Socket reader there is another bump to take. The SocketScan software does not automatically recognise RFID tags in range. The user will have to assign a hardware button to a small application doing the actual scan. This can be any PDA’s hardware buttons. Another option is the Socket Trigger software displaying a small window that is always on top. This window can be tapped to activate the scanner. The trigger software is not such an esthetic solution, but assigning a hardware button will do.

Most of the Flash players however disable the hardware buttons completely or allow the user to remap them to keyboard keys. In fact, all of the 3rd party applications mentioned in the PocketPCMag article as well as the Mobile Bristol Toolkit make Windows functionality for the hardware buttons useless. A big problem, because this renders the SocketScan software and thus the Socket RFID Reader useless!

Fortunately the Standalone Flash Player by Adobe itself is acting like a normal Pocket PC application and does not disable the hardware buttons. It’s the Flash Player 6 though, but for my purposes this is just fine. The recently released mdm ZINC V2 Pocket PC player is able to create standalone Flash projectors and is using the latest version Flash Player 7 for Pocket PC. This application also doesn’t harm normal hardware button assignments. Additionally it comes with a lot of extra Fscommands to extend functionality. But because creating programs for it with the Windows application is quite a workaround, and the Adobe Flash Player will just load ordinary .swf files I did not go for Zinc. Its price tag is also a con.

The SocketScan wedge software allows you to create a prefix and a suffix around the tag’s data when sending it as virtual keystrokes. I chose to wrap it in ‘(’ and ‘)’ characters, for example: (4313750134). With a Flash class listening to keypresses the start and end character can be recognised and the actual tag ID can be used as parameter in other methods.

There has been miscommunication with HP Labs Bristol. Their Mobile Bristol Toolkit has interesting support for use of RFID tags. The editor software allows you to assign actions for RFID tags coming in range of the reader. Also losing a tag (a read tag is going out of range) can get certain actions. Although at first I was told the toolkit needed the ACG CF RFID Reader, when speaking to their technical staff I was told only the next release of their toolkit is going to have support for RFID! This future release, scheduled for around September, is coming too late for me.

I’m currently working on the user interface of my project in Adobe Flash. The Mobile Bristol Toolkit has excellent full screen support. Their Pocket PC client is basically running Pocket Internet Explorer in a fullscreen application. In the HTML page loaded a Flash movie is loaded fullscreen as well (240×320 in my case). The toolkit is able to directly call functions in the Flash movie with its actions on RFID reads.

It looked like the perfect application for a PDA to create graphically interesting RFID applications. The future will not have Flash support. Instead, the MB client software will have timeline support of its own. I can’t imagine the possibilities will be able to meet Flash’ possibilities, but maybe it’s going to be very flexible after all.

For my project hearing this is quite dissapointing. I will have to try to return my reader and will have to look into the possibilities of other RFID readers.

Today I finally received my CompactFlash RFID reader. As I wrote earlier it’s the Multi ISO CompactFlash reader from ACG, that is supposed to work with the Mobile Bristol Toolkit. It came with 30 small round epoxy RFID sticker tags (I-Code SLI).

Although I expected a lot of trouble getting the hardware to work it seemed to go pretty easy. Although I did not get a CD, manual, URL or anything with it, setting it all up went quickly. On the ACG id website a Demo Utility can be found which can be used to test the reader. The manual on the utility (found in the ZIP file) is not too detailed about the available features but it does explain the basics. After clicking one of the two ‘connect’ buttons the ’select’ command enabled me to read out the tags I received, which had unique numbers written to them already.

The Mobile Bristol Editor provides an excellent programming environment to program the client’s behaviour on encountering and losing tags. It’s now time to get the Mobile Bristol client to actually communicate with the reader.

Earlier I wrote about deciding on a platform for my project. One of the reasons to go mobile was to engage a close relationship with the object. The mobile device would be moving towards the object. This way the interaction is directly the opposite of moving the object to the reader. The user is approaching the object instead of moving the object away from himself.

In the end I decided to go for a PDA with a CompactFlash-based RFID reader. It looked like building a fullscreen Flash application for Windows for Pocket PC was the way to go. With the new Flash Player 7 for Pocket PC .flv video files are also playable. Unfortunately no public standalone Flash Player is released by Macromedia. I could go for the Internet Explorer plugin, but at the top and bottom a lot of screenspace is lost because of the menu bars. Next to that it would be pretty distracting and ugly.

Werner Ruotsalainen wrote a great article describing the possibilities for Flash on Pocket PC. Bryht Flash Player looked like the way to go. It comes with a button bar (play, stop etc.), dissapearing when using the program fullscreen. The trial version was not that smooth, so I could not really test if it was worth the money.

To use the CompactFlash RFID reader in combination with Macromedia Flash a bridge would have to be created. Some readers like the Socket CF RFID Reader Card 6E come with keyboard wedge software faking keyboard presses. This solution would not win the beauty contest but would do the job. The Mobile Bristol Toolkit by HP Labs Bristol however provides a superior interface not just for GPS, but for RFID as well! It’s GPS possibilities were used by the 2005 EMMA group project iPACK. For RFID the toolkit should be working with the ACG CF RFID reader, which I hope to receive soon.

Using the Mobile Bristol Editor the software can be programmed to send commands to Flash upon reading a tag and losing connection with a tag. The client software running on the HP iPAQ is not a standalone Flash Player but is using Internet Explorer functionality. In fact, the client is showing an HTML page where a Flash object can be embedded. It is on the other hand able to display the Flash movie fullscreen!

A little annoyance of Pocket Internet Explorer for Pocket PC 2003 is the popup “Press OK to continue loading the content of this page.”. It was Microsoft’s way to work around a patent, explained here, and pops up every time a Flash movie is played. My project file would only be run once, so for me this is not such a big problem. There is a (commercial) client side solution though, called ‘PressOK‘. It quickly removes the popup but you will still be able to see it for a slight second. The Microsoft Knowledge Base article describes a few Javascript solutions to counter this problem. This is a serverside solution, perfect for my project, and also compatible with the Mobile Bristol toolkit.