Terminology (and how sound works)

First, let's start with the basics. A sample can, confusingly, have two meanings. It can either be used to describe a sound clip (e.g. a .WAV file), or one of the individual units of sound that make up a part of the clip. For the latter, a sound sample is typically stored as a 16 bit signed integer. After some processing by the sound software/hardware to provide volume scaling or mixing with other sounds, the sample is sent to an analogue-to-digital converter that translates it into a voltage. This controls the position of the diaphragm in the speaker. In order to generate an audible sound, the position of the diaphragm must be changed many hundreds or thousands of times a second, resulting in it oscillating back and forth. These oscillations set up the required pressure waves in the air, which travel outwards from the speaker and eventually stimulate the receptors inside your ear.

Most sound hardware operates by processing a certain number of samples per second - this is known as the frequency of the sound system, and is measured in Hertz (Hz). Most hardware supports several different frequency settings, e.g. 11kHz, 22kHz and 44kHz. Although at first glance it may appear that the computer is only able to generate sounds at 11kHz, 22kHz, or 44kHz frequencies, in reality many different frequencies of sound can be generated by sending the correct sequence of samples. For example, if the sound system is set to 22kHz, and the sequence of samples 32767, -32768, 32767, -32768, 32767, ... etc. is sent to the hardware, then a constant 11kHz tone will be emitted by the speaker. The tone will be at 11kHz instead of the expected 22kHz because the period (duration) of the wave is 2 samples. I.e. every 2 samples the in-out motion of the speaker diaphragm will repeat (and it is in the in-out motion that results in a sound being produced). If the sequnce of samples had instead been 32767, 0, -32768, 0, 32767, 0 -32768, 0, 32767, ... then the tone would be at 5.5kHz, because the period of the wave is 4 samples. Many other patterns are possible, each providing approximations to different frequencies of sound. The higher the sample rate of the hardware, and the higher the resolution of the samples (e.g. 24 bit instead of 16 bit), the closer these patterns come to representing the real shape of a specific frequency sound wave. One thing to remember though is that the maximum frequency sound you can output will be half the frequency of the sound system, because a period of at least two samples is required in order to represent the essential in-out motion.

Although I've mentioned the period as being the duration of one cycle of a sound wave, it is also used to describe the duration of one sample. For example, at 22kHz, the period of each sample is 1/22050s = 0.000045351 seconds (Note that 22kHz rarely equals 22000Hz. A 22kHz sound system would typically run at 22050Hz, and it is only written as 22kHz as a matter of convenience. Similarly 11kHz is 11025Hz and 44kHz is 44100Hz). This means that, every 0.00045351 seconds, the sound hardware reads a sample from the sound buffer, converts it to a voltage, and sends it to the speaker.

The buffer is another important aspect of the sound system. Rather than constantly pester the CPU with requests for individual samples, the sound hardware typically requests several hundred samples at a time, which the CPU will put into a memory buffer ready for reading by the sound hardware. This provides more efficient operation, as it reduces the number of context switches that the CPU must perform (A context switch - e.g. switching from running an application to running the sound buffer fill routine - will take a finite amount of time. So the more context switches there are, the more CPU time will be lost to performing the switching) Bigger buffers will require less context switches, and so will incurr a lower CPU overhead. However care has to be taken, as the buffer introduces a delay in the sound system - if the buffer is too large the user will notice that a sound is only heard a considerable amount of time after it was meant to start playing. This is more of importance to games than regular desktop use, as games rely upon many sounds starting and stopping at specific times. It is also important for movie playback, to ensure the sound is in sync with the pictures - although if the buffer size remains constant, the delay can be taken into account by the movie player code.

The RISC OS sound system

There are actually two RISC OS sound systems - the older, practically obsolete 8bit sound system, and the newer 16bit sound system. Although I'll be discussing both in this section, the code samples in the next section will focus on the use of the 16bit sound system, as it is the easiest to write for, produces higher-quality sound output, and has better support for mutliple users (via the SharedSound module).

In the beginning though, there was just the 8bit sound system. This is the system that was shipped with the first ARM-powered machines running Arthur. It could be configured to support between 1 and 8 channels of 8bit audio, which would then be mixed together in hardware to produce the two stereo channels output to the speakers. To do this, each channel could be assigned a stereo 'position', which controlled how much of it was mixed into the left and right output channels. Although the sample rate and buffer size of the sound system could be changed, these settings could not be changed on a channel-by-channel basis - they had to be a global change.

The 8bit sound system was based largely around the notion of voice generators - pieces of code (normally contained in a module) which either played back samples from a stored audio clip or generated the data on-the-fly. Each channel would be assigned a specific voice generator, meaning that each channel can only play one sound at once. Sound playback is controlled via the channel interface - so rather than request for a specific sound to be played, you instead issue a standard call to the channel controller to play whatever sound (voice) is currently attached to the given channel. The pitch, volume, and duration of the sound can be specified with the command, although it is up to the voice generator whether it pays attention to those values or not. Although each channel can only have one generator attached at once, there is nothing to stop the same generator from being attached to multiple channels.

Although adequate when first introduced, it's obvious that the old sound system contained several flaws. Only 8 channels existed, and so only 8 sounds could be played at once (unless software mixing was used). It was also a single-user system, as no framework existed for allocating specific channels to specific programs. Furthermore, the reliance upon voice generators made it difficult to implement continuous sound sources (such as music players). Although it is possible to implement a music player as a voice generator, the solution several music players took was to bypass the channel allocation system entirely and instead claim the entire sound system for itself, preventing any other sounds from being played at all (even if the music player only uses a few of the available hardware channels).

The other point to note about the 8bit sound system is that it used logarithmic samples, not linear; although this makes the data more difficult to process, it results in a marked increase in sound quality, as it essentially gives the computer a 12bit sound system.

16bit sound and SharedSound

With the introduction of the RiscPC came a standardised 16bit sound specification, and the hardware to go with it. The main aspect of the specification was the new SWI, Sound_LinearHandler, which allows programs to register their own 16bit sound handler function. This function has the role of filling both the left and right channels of the 16bit sound buffer with data; thus it is limited to stereo sound hardware only. The main drawback of the 16bit sound system is that only one handler can be registered at once - however this was soon resolved with the release of the SharedSound module, which is now the standard interface to the 16bit sound system under RISC OS.

The SharedSound module operates by registering its own handler (via Sound_LinearHandler); it then allows clients to register as many sound handlers as they want, via the SharedSound_InstallHandler SWI. Each time a buffer fill request is received from the Sound module, SharedSound will iterate through the list of client handlers and call each handler, allowing them to write whatever data they want to the sound buffer. Because the same buffer is shared between all clients, each client must correctly obey the mixing flags that SharedSound passes to it, to ensure data from other clients doesn't get overwritten. However the use of linear 16bit sound samples makes the mixing process trivial to perform.

Apart from providing shared access to the 16bit sound buffer, SharedSound also provides useful information about the sample rate of the buffer, in particular fractional step values to convert from the client's sample rate to the buffer rate. It also allows the volume of each handler to be changed individually (although it is up to the fill code to perform the required volume scaling). It also allows several different types of handler to be used - standard handlers that can be run from an interrupt handler, callback handlers that perform processing in a callback (which means they can take longer, as interrupts will be enabled), and process handlers (which are called in a callback after all other handlers, to allow for any effects to be applied to the final sound buffer data). In my experience, even with relatively simple buffer fill code like the one in the example below, if you're playing more than a handful of sounds at once you will get better performance by switching from an interrupt-based fill routine to a callback-based fill routine.

The only downside to the SharedSound module is that the latest version is only available if you own RISC OS 6. If you don't have RISC OS 6 (Or any earlier version of the module shipped with RISC OS Select/Adjust), then it looks like the most recent version available on the Internet is version 1.04, available from here on Castle's website. For documentation, the best source would appear to be the 'OS' StrongHelp manual, available here on the StrongHelp site.

A sample simple SharedSound sample player

I'll now go into detail about how to write your own sample player, using SharedSound. The player will load one or more WAV files from disc and play them back all at the same time - demonstrating how to read (simple) WAV files, how to interface with SharedSound, and how to perform relatively simple activities such as playing multiple sounds at the same time, at their correct sample rates. The code could be easily expanded to support other features, e.g. looping, staggered playback, individual pitch/volume controls for each sound, etc. - basically everything you need for a simple music player or computer game.

Download the code

To run the sample you will need RISC OS 3.5 or above, a copy of the SharedSound module, and some WAV files. The included makefile is suitable for compiling the code with GCC.

The code is split into four sections, which will be explained below. It is a mixture of C and assembler; C for file reading and general control, assembler for buffer filling and some error handling. The code could have been written as a mix of BASIC and assembler (or in pure assembler), but processing the WAV files in C is much easier than in BASIC or assembler. If you really wanted you could use C for the buffer filling code instead of assembler - but that would require extra wrapper code to get the CPU/stack in the correct state for the running APCS code, as well as extra care when writing the code to ensure certain instructions (e.g. floating point) aren't used. For a simple player like this, it's a lot easier and safer just to write the buffer fill code in assembler.

1. WAV reading

Each WAV file is read into a sound struct by the load_wav() function. This function will take the filename and sound pointer as input, and attempt to convert the WAV file into a sequence of stereo 16bit samples (stored in the correct format for placing in one of SharedSound's buffers). Although the code is by no means perfect, it should be able to understand almost any uncompressed 1 or 2 channel WAV file you throw at it. If you're interested in writing your own WAV reader, then you're out of luck, because I've lost the link to the original page I used when writing the code. All I can do is point you to Wikipedia and let you struggle by yourself if one of the sites they link to gets it wrong!

2. SharedSound initialisation

This is performed by the init_sharedsound() function. After the buffer fill code has been registered with SharedSound, init_sharedsound() performs two other important tasks - it registers a couple of error handlers (more on that later) and reads the current sample rate. It uses the sample rate returned by SharedSound to calculate the playback rate of each WAV file. This is necessary because the WAV loader doesn't do any sample rate conversion. Instead, each sound is given its own playback rate.

3. Playback

On the C side, not much occurs during playback. The code merely waits in a while() loop until it detects that all samples have reached their end. This means that playback is entirely under the control of SharedSound - and our buffer_fill() code.

Each time the buffer_fill() code gets called, it processes the sound list, adding each sound to the buffer one at a time. I've deliberately left the buffer fill code quite simple, so that you can see how it works. For one, it ignores most of the information SharedSound supplies it with, such as sample rates and volume levels - because it should already know the sample rate to use, and no volume scaling is performed. Secondly, no math overflow checks are performed on the samples as they are added into the sound buffer - this will result in clicks and other noise if you play too many loud sounds at once. Extending the code to support volume scaling and/or overflow protection is fairly trivial.

Note that because of the way sound works, in order to play two sounds at the same time we merely have to add the two samples to each other. This will only work with linear sound samples however - if this code were for the 8-bit sound system then it would be much more complicated (which is one of the many advantages of using the 16bit sound system instead).

It's also worth pointing out the use of R7, R12, and R8 in tracking the playback position. R7 and R12 contain the playback position of the sound; R7 is the whole part (i.e. number of samples played) and R12 the fractional part. R8 is the sample rate. This is calculated (in init_sharedsound()) as the ratio of the WAV sample rate to the SharedSound playback rate. So if you were playing an 11kHz WAV file into a 22kHz SharedSound buffer, the ratio would be 1:2, or 0.5. This ratio is stored in R8 as an 8.24 format fixed-point number (8 bit whole number, 24 bit fraction). So the 0.5 of the example would become 8388608 (Or 0x100000 in hex). R12 is also stored in 8.24 format; this means that, each time the playback position is updated, the upper 8 bits of R12 are added to the lower 8 bits of R7, and then cleared from R12. R12's sole purpose is to track the fractional part of the sample position.

Note that R7 is saved in the sound state, but R12 is not. This does mean that if any value is left in R12 after each buffer fill, it will have the effect of stretching the sample slightly. For example if there are 100 buffer fills per second, then at most the sample will be stretched by 100 samples per second, making it 1% longer and 1% lower in pitch. If I were writing a music player then I might store the value of R12 between buffer fills; but for a simple sample player there is little point, as the effect is minor and not guaranteed to occur anyway.

4. Shutdown

Shutdown is handled by the aptly-named shutdown_sharedsound() function. This function unregisters the SharedSound handler (if it is still installed, as an error may have forced it to be removed), and disables the ErrorV handler that was installed by init_sharedsound(). The ErrorV handler is used for error handling, which is discussed below.

Error handling

Error handling is a very important part of audio playback under RISC OS, especially if your buffer fill code runs from application space or relies on data stored in application space. If your program exits in an unexpected way then there's a chance that the buffer fill code will be left active - at which point anything could happen from random data beginning to play or the machine locking up entirely. After struggling with error handling for some time, I've come across a solution that appears to work under all circumstances.

Firstly, a C atexit() handler is used to catch all cases of the C program terminating under the control of the C library. This handler will get called if the program exits via exit(), abort(), or by returning from main(). In our case, shutdown_sharedsound() is registed as an atexit funciton.

Secondly, an assembler function is attached to ErrorV, the vector that will be executed whenever the OS is made aware of an error. This is necessary to trap the cases which atexit() does not - for example more serious errors such as data aborts or undefined instructions. The assembler function that gets registered (error_handler()) disables the SharedSound handler, but, importantly, does not disable the ErrorV handler. This is because after testing it on my Iyonix I found that the machine would lock up completely on error, most likely because it does not support an error handler removing itself. Manually removing the error handler isn't deathly important anyway, as the OS is capable of removing it itself when the application exits.

Multitasking

A couple of extra considerations are needed for multitasking programs. Firstly - don't expect to be able to play sounds in the WIMP using a player in application space. This is because of how RISC OS mainpulates the memory map to allow each program to run from the same &8000 base address. Secondly - don't expect to be able to do it from a TaskWindow either (hence the check near the start of main() in the example), again for the same reason as above. If you want to play sounds from within the WIMP then you'll have to rely on buffer fill code stored in the module area (ideally inside a module rather than as a random piece of floating code), and store your sample data in a dynamic area or in the module area.

Interrupts

One important thing to remember if you're doing more complex work with sounds is that all buffer fills occur in interrupts. This means that your program doesn't know when they will occur; if you have some C code that adds and removes sound effects from a list of effects to play, then there's a chance the buffer fill code will get called in the middle of your code updating the list. You should design your code with this in mind. For example, if you are swapping a long sample with a short sample, and you update the data pointer before updating the playback position or sample length, then there's a chance random data will be played instead of sound data. The best course of action in this case is likely to be to temporarily set a 'paused' flag for that sound, or set the data pointer to 0 (and have the buffer fill code interpret that case as there being no data to play). If your sample playback list is constantly changing then you might want to keep two lists - one which is in a safe state and is read by the buffer fill code, the other which is in a volatile state as it is constructed or updated. After a list has been constructed the 'active list' pointer will be swapped to point at the other list. If you take this approach, remember that you will have to store the playback position for each sound somewhere sensible (i.e. not in either of the two lists)

For example, the game I'm currently working on, DeathDawn, uses three structures to manage sounds: Each sample is stored in a sample structure. This contains the raw sample data, length, and sample rate of that data. Each object in the game has a fixed number of sound slots; each slot can reference a sample, and contains the playback position, volume and pitch modifiers, as well as pause and repeat flags. Although there may be 30 objects in the game world with 4 sound slots each, there may only be 4 slots which are both in use and in earshot of the player. For this reason, two lists of active sounds are kept. These lists only contain pointers to the sounds that should be played. As well as the sound pointer itself, each entry also contains the final volumes of the left and right stereo channels, and the fully-adjusted 8.24 fractional step to use for playback. At the end of each frame, the active list that is used by the buffer fill code is swapped with the list that has just been assembled by the C code. Although this solution solves many problems with handling sounds it is still not perfect, for special code must be used to delete sound structures (to ensure they are not referenced in either the of the active lists), and if a sample is swapped for one of a different sample rate then there's a chance it will be played at the wrong rate for the duration of one buffer fill (although this can be fixed by moving the sample rate calculation into the buffer fill code). The reliance on active sound lists also introduces a delay of a few centiseconds between a sound being scheduled for playing and playback to begin.

Next time...

The next article, due sometime before judgement day, will deal once again with the topic of random map generators, as well as provide numerous examples of uses for the different container data structures that were discussed last time.

3 comments in forum ]]> Mon, 17 Mar 2008 12:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1171.html http://www.iconbar.co.uk/comments/rss/news1171.html
But whose birthday is it?

It's ours, of course!

Ten years ago today, Acorn Arcade officially opened to the public, as this handily-archived newsgroup posting shows. With Alasdair Bailey and Graham Crockford at the helm the site soon increased in popularity enough to warrant moving to its own domain name, acornarcade.com, and eventually to its own dedicated server, owned by our benevolent dictator Rich Goodwin. A few years later Acorn Arcade gave birth to The Icon Bar, which then in an ironic turn of events absorbed the content of Acorn Arcade in late 2006, in order to give both sites a much-needed overhaul.

But what does all this mean?

Well, despite the fall of Acorn, changing home, giving birth, and then getting eaten by our child, the award-winning Acorn Arcade is still here, and we hope that we'll be able provide you with news, reviews, downloads, help and support for at least another 10 years, even if we're not posting as many articles as we used to.

And speaking of articles, we've got a little surprise for you. When migrating Acorn Arcade's content into The Icon Bar's new article system we found a couple of folders on the server that looked like they hadn't been touched in years. And inside those folders we found some relics from times past - Acorn Arcade news archives stretching back to April 1998, and previews of MicroDigital's Mico and Acorn's ill-fated Phoebe machines. Since we were on a tight launch schedule for the new-look Icon Bar, the decision was made to hold these old articles back. After all, no-one except us knew about them. Predictably, we promptly forgot about the forgotten folders, until the topic of our 10th anniversary popped up in the staff forum. Luckily we were able to take a break from our rigorous schedule of procrastinating podcast 6 in order to import the articles in time for today.

Also, with a little help from the Internet Archive Wayback Machine, we've managed to get a couple of screen shots of previous designs of the site, which you may find interesting. Or not.

Forgotten news archives

1998	Pre-July	July	August	September	November	December
1999	January	February	March	May	June	July-November

Forgotten previews

Alasdair's Phoebe preview
Foggy's Mico preview
ChopX Preview
King and Country preview
Super Snail 2 preview
Toy Chronicles preview
Wubble preview

Other forgotten articles

ART thingy - Who are these strange people?
ART attack
ChiBER - RISC OS in a PC box
What's happening on csag?
Doom and Heretic Updates
Heretic upgrades
1998 Midlands show report
NetStation gaming update
PushyII for the PlayStation

9 comments in forum ]]> Tue, 29 Jan 2008 00:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1166.html http://www.iconbar.co.uk/comments/rss/news1166.html Well it's that time of year again. It hardly seems like a full year since we last wished all our readers a very Merry Christmas. Yet again, most of you are probably too busy downing the bottle of whatever-it-is that you found in the back of the cupboard, and stuffing yourself with mince pies to notice this post, but I'm still going to say it anyway. Well it's that time of year again. It hardly seems like a full year since we last wished all our readers a very Merry Christmas. Yet again, most of you are probably too busy downing the bottle of whatever-it-is that you found in the back of the cupboard, and stuffing yourself with mince pies to notice this post, but I'm still going to say it anyway.

Merry Christmas to all our readers, and we hope you have a wonderful new year - whatever you're doing and however you're celebrating.

Links:
Christmas (Wikipedia)

1 comment in forum ]]> Tue, 25 Dec 2007 00:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1165.html http://www.iconbar.co.uk/comments/rss/news1165.html A guilty secret: limited though they were, I used to love working with early Palm and Psion PDAs. Neal Stephonson wrote in his novel Cryptonomicon... A guilty secret: limited though they were, I used to love working with early Palm and Psion PDAs. Neal Stephonson wrote in his novel Cryptonomicon:

Eb is doodling on one of those little computers that uses a stylus so that you can write on the screen. In general, hackers don't use them, but Eb [...] wrote the software for this model and so he has a lot of them lying around.

Fast forward a few years, and along comes the Nokia 770 Internet Tablet: a small device with a touchscreen, but updated for the 21^st century with wifi Internet access, a widescreen, full colour display, a proper Web browser and bluetooth connectivity. I'd looked at the proliferation of Windows-based PDAs over the years and they'd never appealed. Where the Palm and Psion devices felt like they'd been designed from the start with mobile computing in mind, WinCE always seemed like a big OS shoehorned into a little device, and wifi an afterthought if available at all. And don't get me started on small keyboards after the disaster that was the Psion Revo. Maybe this Linux-based device could put the fun back in to computing?

Then, notice the price tag and fast forward some more until we get to the part where I buy one off eBay.

What's in the box?

Opening the box, the device looks tiny. The word "tablet" and the 800x480 resolution (and my inability to read metric specs properly) meant I was expecting something more, well, tablet-y. Maybe the size of those 7" picture frames, which have a similar resolution. Turned 90 degrees it's about as tall as your average PDA, but maybe a bit slimmer across. The Nokia site still lists the N770 RRP as £245, which seems a bit much for what it is, but on eBay they usually go for around £90, and I got mine for eighty quid plus postage. Amazon have just brought their price down to £129.99 "with Navigation Kit" if the thought of eBay-ing puts you off. Around £100 seems like a sweet spot, as it compares favourably with other boys toys of a similar size and complexity such as a mobile phone or MP3 player. Anyway, the size was a slight shock, but given that it'll be spending most of its time in my hands or pockets, not a particularly unpleasant one.

Only glancing at the manual long enough to get the battery compartment open and occupied, I switched the device on, and... well, there were three things going in that I decided the device would live or die on: connectivity, browser, and the quality of the screen. The latter seemed a tough one now that the device was smaller than I'd expected, but... wow. Some of the lettering can be tiny, but it's never less than readable thanks to a combination of a very sharp screen and good anti aliasing. Obviously if you squint to read the newspaper then this device won't do you any favours, and like all LCD displays direct sunlight can be a problem, but sitting on the sofa it's actually... pleasant. That's not usually a word I associate with computers.

Nokia N770 (hand: model's own)

Set-up was similarly un-troublesome, consisting of a couple of general questions and brushing aside a request to pair with a mobile phone, then I'm straight into the desktop. Status icons at the top, program shortcuts down the left, and a big space in the middle for some built-in widgets - I kept the RSS feed reader, clock, and Internet radio player, but swapped the Web search for weather reports.

Continuing a trend, getting the device connected to the Internet is done just right. If you're not connected and try to make a request for, say, a web page, the device will just connect to one of your saved network connections in the background, then get on with downloading your content. Contrast this with my Windows laptop, when the connection drops: a browser error page and a minute or two to reflect on how bad Windows is at figuring out whether it's on the network or not, usually followed by my having to do a manual connection repair and uttering a few choice words. On the N770 if you bring up the list of available access points, it'll keep refreshing the list so you can see any changes (or can use it to sniff for open networks as you wander around) - on Windows, it generally just tells me there aren't any for the first couple of times I press the refresh button.

The wireless networks in my area (click for bigger)

The browser itself is Opera-based, which from its early days was designed to be small and quick but without compromising on features. The device is so small that it's easy to underestimate it, but you get proper CSS, Javascript and even Flash 6 support. Some pages look dead on - BBC news, for instance, works right down to the ticker near the top. If you want to access the RealMedia content, then it won't work in-browser, but requests for audio or low-resolution video will spark up the device's media player just in case you can't do without News 24. Other sites need a little horizontal scrolling, and some can look a bit scrunched - ours, for instance, at least until I hacked in a little Javascript to hide one of the side bars.

It's not perfect: it has a tendency to crash when it runs out of memory, and some badly-designed sites - the naff-textured, advert-heavy Ain't It Cool News for instance - can push it over the edge. Empire's website (to continue the movie theme) has some minor alignment issues with some images, but at least it doesn't cause the browser to barf. Some Flash sites require a later version without backward compatibility, so no Weebl and Bob. On the whole though it handled pretty much everything else with some style, including AJAX-based sites.

BBC News website (click for bigger)

So far, so good

The important bits out of the way, time to look at some of the secondary considerations, like battery life and other software. The battery is supposed to last 3 hours, okay compared to a laptop, maybe less so compared to a mobile phone or MP3 player. However, I find I only have to recharge it once a day, partly because I usually turn it on and off as I need it, and partly because when I leave it lying around switched on it has a fairly aggressive power saving mode. Put the protective metal half-shell back in place and the power saving is even better - I've regularly left it switched on in my pocket for 24 hours, and up to 36 or 48. It was weeks until I heard this strange noise coming from my pocket, which puzzled me for a while because I honestly didn't know what the low battery sound was.

Considering it's using wifi constantly when in use, and bluetooth too at times, battery life is more than adequate. My laptop needs charging after two hours no matter what. Being an eBay purchase obviously the charger isn't a UK one, but the ickle tiny dual voltage power adapter came with an equally diddy converter. It does point sideways when plugged in, but at the end of a four-way block it's not going to trouble any other power bricks. Your mileage may, of course, vary.

If the device had to live or die on just the Nokia software, however, I don't think I'd have been quite so happy. It has some useful, if not exactly spectacular apps - email and messaging, the rather dull-looking media player and RSS feed reader, a PDF viewer, three worthy but uninspiring games (I like the well-featured Mahjong, but Chess on a mobile device?) and so on. There's no real business software (e.g. a Word compatible word processor), so it's unlikely to please the Road Warrior crowd. Going into the application manager to check for new or updated software is similarly unlikely to light your fire either - a few themes, a simple media streamer, and not a great deal else.

However, this is a Linux-based machine, so there's a happy end to this story. The browser home page (again, slightly dull and grey) has a link to the "Tableteer" site, and a little hunting around gives a link to Maemo.org... or, like me, you can just Google for "N770 software" before you buy the thing and go in well informed. Maemo.org's downloads section is similar to the Firefox extension website, in that you'll probably spend hours scrolling through thinking "ooh - want that. And that. And that...". I've installed tons of useful stuff, only to find that I've still only used up two thirds of the main "drive" - app sizes here are measured in kilobytes, not megs or gigs.

Killer apps include Maemo Mapper, which is a GPS mapping program. eBay had also furnished me with a solar-powered bluetooth GPS unit for under thirty quid, so I was eager to give this a try. At first I just got a blue dot in the middle of a blank page, but on reading the forums that's because for copyright reasons it's not provided with any maps. What you do is give it a cunningly-modified URL to your favourite mapping service - e.g. Google Maps - and it downloads the images you need. Similarly you can "download" driving routes, and even a speech system. Obviously it's unlikely that you'll have decent Internet access in your car (unless your 'phone is better than mine), so you can pre-cache the bits you need, or just download a whole square between two coordinates. When you're done, Topografix-format XML files of your tracks can be downloaded for use in other software, handy for outdoors types.

Maemo Mapper in action (click for bigger)

If that's not your thing, then the more advanced media players might take your fancy - UKMP (Urho Konttori Media Player) is a swish iPhone-style music player; album covers on a black background can be navigated using "kinetic scrolling", where a flick of the finger will set the screen scrolling until it hits the top (or bottom) of the list, or it runs out of steam. Kagu does a similar kind of thing, but with a different layout. Canola goes even further, presenting a media centre-style interface to music, video and pictures, although you'll need some software installed on a "proper" computer to stream the content.

For most people picking from the myriad Shoutcast streams available will probably be enough (who knew there was an entire radio station devoted to just Beatles songs and cover versions?), but I wanted my own music. TVersity is a free media steamer for Windows, and *nix types can probably work something out with gstreamer, but in the end I plumped for installing GNUMP3d on my Linux file server to handle audio streaming. If you just want to play MP3s from the device itself, the N770 only comes with a 64MB MMC Mobile card which won't hold a great deal, but you can buy one up to 1GB for about a tenner these days - or 2GB if you install a third-party kernel.

Ports of AbiWord and the GPE suite of programs will take care of most office software requirements, but that was of little interest to me personally - getting VNC and SSH working mattered much more. I will say that the PDF viewer acquitted itself well, handling a large, graphically intensive magazine file from the Nottingham City Council website pretty flawlessly, if slowly at times.

Installation will be fairly familiar to anyone used to a Debian-style Linux distro such as Ubuntu - .deb packages can be downloaded and installed (or not, if you hit a dependency issue) or you can manually edit the repository list and grab them from the application manager. Properly packaged software usually comes with an install link that will take care of all of the above for you however.

Signs and portents

There are many clues that this device is not packaged in the usual corporate way: it's a hacker's toy, not a business tool. You don't need to purge the browser bookmarks to get rid of all the upsell "services" that usually get shovelled there - it's useful stuff like BBC world news, Lonely Planet travel guides, a Google directory with just search, mail and news, and links to the two main N770 sites. The example images are Discovery Channel shots of monkeys and sharks, instead of bland copyright-free landscapes. Even the example music features someone who can hold a tune, even if it's a bit Country for my tastes. OK, so it starts up with that jingle and a faintly vomit-inducing picture of a baby's hand reaching out to an adult, but all in all, not too bad.

Then there's Red Pill mode.

To make certain changes on the N770, you sometimes need root access. To do this you simply go into the application manager, bring up the screen to enter a new repository, and for the web address just type matrix then click the Cancel button. You'll then get a choice - "Which pill?". Red grants access to the inner workings of the MatrixN770; I used it to install SSH. Blue pill puts everything back to normal. SSH installed not only the client but also the server - so the addition of an xterm client means I can log into localhost as root (don't forget to change the password!), or even log in over the wireless network from my desktop machine. As the N770 is a Linux computer under the hood I could easily go into /etc, add a CIFS module and put in a couple of Samba shares in the fstab file. I have to run a small file once the device is on the network to activate these shares, but then I have access to several gigs of storage on the fileserver as if it's on the device itself.

Laptop replacement?

If you're one of those people with a huge desktop replacement laptop, playing all the latest games or sitting in a coffee shop all day writing haiku, then obviously the N770 isn't a direct replacement for that - and you've probably got a tiny penis, as we've established on the forums.

But I've got a perfectly good desktop machine, thanks, and found I only used the laptop for occasional stuff: checking news from the sofa while an advert break is on; or sitting on something more rounded and with a base made of porcelain every morning. Even a small laptop is a bulky thing to carry around with you everywhere just in case, so having something pocket-sized means when you gotta go, you can just go.

Without a stopwatch on it, it's debatable whether it starts up any faster than bringing the Windows laptop out of hibernation, but there's a difference between Windows coming out of hibernation, and Windows deigning to allow me to access a web page. As the N770 has no moving parts and can safely be started up while I'm in transit to the smallest room in the house, it wins in the end anyway. Then if you leave it on in powersave mode, it comes back instantly.

To use a real world example of where a mobile device comes in handy, I recently noticed that the server was running under heavy load via one of my diagnostic webpage. As I was in bed at the time, I used the N770 to SSH into the server, used top to see that the spam filters were running a little hot, tailed the mail server logs, and found that someone was sending spam with one of our subdomains in the "from" address so we were getting all the bounces. I just changed the mail alias file for that subdomain to remove an errant wildcard address using vi, and all the bounces got rejected without troubling the spam filters.

I could have got up and used the desktop machine; I could drag the laptop to bed every night, but the N770 was just there, and handled all of the above just fine. I can also VNC into machines - to start (e.g.) the Bioshock demo downloading first thing in the morning, or shut down the desktop machine when it's finished doing a backup last thing at night - or just pull up the wiring diagram of the internal USB connector when fitting one of those Matrix Orbital LCD displays. While I doubt that these abilities are unique to this device, having properly integrated wifi and that widescreen display certainly make it an ideal tool for the job.

VNCing into my desktop machine (click for bigger)

I carry a Leatherman multitool on my belt, which isn't going to replace a proper well-equipped toolbox - there's no power tools, and it's not heavy enough to hit things with - but it's good enough for most day-to-day jobs and it's always there when I need it, so it makes my life that bit easier. The N770 can be described in the same way.

Problems

It's not perfect, of course - but writing about the downsides of the device have taken me longer than the rest of the article put together. This could in part be due to the honeymoon period of first getting something new and shiny, but even after a few weeks it's hard to complain too much. There a few annoyances, but no real show-stopper I can really rip in to.

When you switch on the device, it needs to finish its boot sequence properly: if you jump in early and start trying to connect to your wireless network, there's a chance the device will think it's gone pear-shaped when it notices the extra load and just reboot itself. Similarly, if it runs out of memory - most notably websites heavy with graphics - it'll quit the program you're using, pointing to there being not quite enough memory under the hood. The N800 appears to have double the N770's (128MB over 64MB), which might add credence to this. These are problems that can't be completely cured as such, but with a bit of common sense can be avoided most of the time - especially as there are a number of programs available to add memory and load indicators to the top bar. In normal use however, having three or four programs running at once hasn't caused me problems.

I found the USB connectivity pretty useless - it only allows you to browse the mini SD card, if fitted. And if you're connected, you can't see the card from the device, so you have to keep disconnecting and reconnecting the cable to transfer (to take a random example) review screenshots. Uploading the images directly via the web browser is probably more sensible. It would have been better to remove the USB port, use the extra space to fit a proper-sized SD card port (so it can be taken out without having to clip the adapter in place, and coincidentally read my camera's card directly), and leave the USB cable out of the box. If you're really worried about it, perhaps use the savings in component costs to bundle a dirt cheap card reader: it'd be more convenient - at least this would show up as a removable drive, allowing the user to do the safe removal thing under Windows. Or hey, fit a USB input instead and allow me to access USB devices instead of just a few bluetooth ones, and I could really get rid of my old laptop (although I shudder to think what that would do to the battery life).

One of my personal disappointments was with text area input in a web browser, but I've been in two minds whether to mention it. As well as the usual hunt-and-peck small on-screen keyboard for use with the pen, the application of a meaty thumb in the right place brings up a keyboard that covers most of the screen, allowing me to jab away with my fingers - or holding the device in both hands, type using both thumbs. This works a treat for entering commands over SSH or even typing up notes on a bus, but failed dismally in the text area box in the web browser when replying to comments on our forums. It would either result in small comments vanishing, or large comments repeating lines at random, plus formatting quoted text to a fixed number of columns. The latter could be solved by some pre-posting edits, but it was annoying, as this is my preferred method of text entry. I eventually got around it by installing the latest Gecko engine. However, installing an alpha copy of the latest Mozilla code means a slower, more memory hungry and crash prone browser, so it's not ideal for normal browsing and I find myself switching back and forth between this and the original Opera code. OK, this can be done fairly easily from the browser itself, but it's extra hassle. The reason I'm not sure whether to mention a potential show-stopper? It worked fine on my own "contact me" web page, so although the browser shouldn't have this problem, we might be doing something to contribute to it - possibly some CSS in the text area definition that caused it to go screwy. It's certainly something I'll be looking in to.

The only other problems I had are trivial. For instance, the onscreen keyboard suggests words from the dictionary, and learns as it goes along; this leads to the nice, massive space bar getting much shorter, and I have occasionally ended up with sentences like "I america not happy" until I learned to use the far right of the space bar at all times. Oh, and as it doesn't know SSH passwords from any other text, it's memorized my root password and happily suggests it to me all the time (lucky I have heavy firewall rules to lock SSH down, so I should be OK even if the device is stolen).

Moving the desktop plug-ins around can be painful, as they can't overlap - but you can't, for instance, change the number of columns on the weather app to make it fit better until it's been placed on the desktop, which is a chicken and egg situation. However, once in place I never have to touch them again, and you don't really, truly need anything on the desktop. It does point to some apps being so simple that, for instance, between sessions the screenshot hack didn't notice that there were already screenshots in the images folder and overwrote some of my killer illustrations for this review. I had to remind myself that it was free, third party, and takes up about 1K of space, so just maybe it's a bit much to expect more.

The Desktop Applications in All Their Glory (click for bigger)

And as many third party apps aren't specifically designed for the device, it can be slightly hit-and-miss figuring out what hard button does what - for instance, the volume rocker is used as the zoom controls in certain apps. Having casually thrown the manual aside from the beginning, I had a bit of an ID-10-T moment when I couldn't find any on-screen cursor keys, which means I couldn't use the shell history function when SSH'd into the server. I'd become so used to doing everything on-screen that I had completely ignored the large D-pad on the left side of the device, which is the bloody obvious cursor key combination. I'm still not sure what the button in the middle is used for though; the button that does "back" in the browser looks like a reload button; and I've never felt the need to use the home or menu buttons, if indeed that's what they are. The full-screen key, however, gets a lot of use, and is just in the right place for tapping with your left index finger when you're holding the device with both hands, so no complaints there.

What else? Clicking on the desktop RSS headlines and summaries opens up the same data in the RSS feed reader, where I'd prefer it to go straight to the web page. Oh, and when the metal half-shell protective case is in the storage position (on the back, rather than covering the screen), it covers the little hole that holds the pen, so you have to keep hiccuping the device out of the shell to store and remove the pen. Such a discreet, wifi and GPS enable device is crying out for a NetStumbler implementation, but I can't find one. Yes, I'm really struggling for negatives now. Does being able to reskin the entire device, including sound effects, as a Star Trek LCARS device count?

Conclusion

We debated in the staff forum why I bought this device in particular - some of the points made in this review might seem to be extolling virtues that might be present in many, if not all, mobile devices. Yes, I love the fact that, on visiting my parents' house, I don't have to get out of my chair to tell them the prices and availability of books on Amazon, and that I can pull this device out of a pocket and start taking notes even on a cramped bus, but we're probably not talking about unique selling points of the N770 only there. This is in part because I was looking for something more convenient than my laptop rather than looking for something to review, so I'm sharing some of these experiences to maybe whet a few appetites (and on seeing the web browser in action, I have at least one convert). I just found the makers of this particular device have made many sensible decisions, most notably with the screen quality, connectivity and battery life. Text entry too, apart from the hiccup with our forums. One option I haven't mentioned is the handwriting input, which uses proper letters rather than some obscure shorthand, and with enough examples that it probably doesn't need training - but if it does, you can do this on the fly without quitting and reloading your current program or document. Or that, when you connect to a wifi network, the RSS feed list on the desktop notices and automatically downloads the latest headlines in the background. It's touches like that which make this device so nice to use.

This is a device that takes a little effort to get just how you want it, but it amply rewards that effort. On one hand, it's a capable multimedia machine that has a solid, user-led software base if you just want to play. On the other hand it's a pretty straight Linux computer that's infinitely hackable, without the constant frustrations and work-arounds that you'd face with an iPhone or a PSP. Being a bit cheap I don't think I'd pay top whack for this, or the updated N800 (with built-in webcam - woo!). However, the price I paid is a bargain for the amount of use it now gets, and I wholeheartedly recommend picking one up.

10 comments in forum ]]> Tue, 16 Oct 2007 17:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1164.html http://www.iconbar.co.uk/comments/rss/news1164.html Parmesan is a new client for the popular MSN Messenger network, developed by Christian Ludlam. Released as freeware, the software brings new features to instant messaging on RISC OS. Chief among these are display pictures and nudges. The software also doubles as a versatile viewer for arbitrary XML files. Parmesan is a new client for the popular MSN Messenger network, developed by Christian Ludlam. Released as freeware, the software brings new features to instant messaging on RISC OS. Chief among these are display pictures and nudges. The software also doubles as a versatile viewer for arbitrary XML files.

RISC OS has seen the release of several MSN Messenger clients over the years; two free clients (Natter and Messenger) and R-Comp's commercial product, Grapevine. A few years ago all of these clients fell foul of an upgrade to the MSN protocol which stopped RISC OS users from being able to connect to the chat network. To many users' dismay, RISC OS was left without a free MSN Messenger client, as only Grapevine was upgraded to support the new protocol.

Happily, this distressing situation has, at last, been rectified with the release of Parmesan. In this article I'll take a look at Parmesan describe my experience of it. Finally, as a Grapevine owner, I will compare Parmesan with the commercial Grapevine application.

Signing in

Parmesan was first announced less than a fortnight ago on the Zap technical mailing list, in a call for beta testers. I responded and have been running it ever since. When first run Parmesan presents the user with a simple dialogue box enabling the MSN account details to be entered and lets the user sign in for the first time (figure 1).

Figure 1
"Sign in" dialogue box (RiscPC, running Select 3)

It looks as though Parmesan can cope with multiple users' MSN accounts although I have not tested this myself. Signing in is extremely straightforward and I was quickly presented with a list of my friends; online and offline (figure 2).

Features I like

It is immediately apparent when you start using Parmesan that a lot of thought has gone into the user interface. Looking very much in sympathy with the RISC OS desktop's general design paradigms, it is extremely clean and intuitive. It is uncluttered and has no unnecessary or confusing buttons and toolbars.

Figure 2
Friends window on RISC OS 5

The other thing that struck me when I first signed in was that the software supported "display images", which are little icons, individual to each user. Parmesan is, to my knowledge, the first RISC OS MSN Messenger client to support the feature. Initially, users are given the Parmesan cheese icon as their display image. It very simple to change the image and I soon had a little drawing of a car, created in ArtWorks, as my display picture.

Nudges allow people to attract the attention of the person they are conversing with, by bringing your chat session back to their attention. It is of particular use if your communication is urgent. The chat window (figure 3) has a pane to the side of it which shows your correspondent's display picture and offers two buttons. One of the buttons is for selecting a graphical emoticon and other lets you give your friend a nudge.

Figure 3
The chat window

When I first discovered that Parmesan supported nudges I was a little dubious because I generally dislike things that interfere with my desktop. After experimenting with the feature, I can report that it has been implemented pretty nicely. When you are nudged by a friend, the chat window is brought to the front of the window stack and given a little shake. In my experience, this succeeds in drawing attention to the window without interfering too much with the desktop. The nudged window does not steal input focus, so you can continue typing into another window on the desktop as you receive a nudge. It might be nice to have the option to disable the feature, especially if you suffer any particularly enthusiastic friends.

The text input box at the bottom of chat windows is a multi-line text area. This means that there is plenty of room to read over what you've typed in a long message. It also lets you send multi-line messages. Pressing the return key sends the message, while shift+return starts a new line within the same message.

I very much like Parmesan's whole interface. It takes a few liberties here and there, such as not using the standard RISC OS window resize furniture, however Parmesan manages to pull it off without jarring with my delicate sensibilities. The content of the windows, such as the friends list, is well layed out and has quite a polished look.

Parmesan has very advanced configuration functionality. All of the things Parmesan displays, such as the friends list and chat conversations are represented internally in XML format and they way they are displayed is governed by CSS files. This makes it very easy to restyle the the way conversations are shown. For example, if you want to adjust the gap between successive messages in chat windows you can change the value of the margin-top property for the message class in the Chat CSS file.

Things Parmesan doesn't do (yet?)

File transfer is high on Christian's TODO list. I'm told that it works at the moment but there is no front end for the feature so it is disabled by default.

Once you've set up a personal display picture, if you try Parmesan on another computer you need to set the display picture again, because it is not stored on the server. I am told that it is possible to store your picture on the server with later versions of the MSN protocol.

Parmesan in action on a VirtualRPC powered laptop

The MSN Messenger protocol allows for hand drawn messages to be sent, as well as text messages. Parmesan supports the display of hand drawn messages but does not yet cater for sending them. A screenshot showing Parmesan displaying a hand drawn message is available on the Parmesan site's screenshot page.

There are several other rival instant messaging networks, for example ICQ. Only the MSN Messenger protocol is supported supported by Parmesan. This is no issue for me as all my friends use MSN Messenger.

Beyond instant messaging

Aside from being a perfectly good MSN Messenger client Parmesan is also a useful application to have at hand, even if you have no interest in instant messaging. It can be used to display any XML files you might have. Dragging an XML file to the Parmesan iconbar icon causes it to display the XML file's contents raw. If you then drag a suitable CSS file to the XML window it is styled neatly.

By default, Parmesan logs conversations in Choices:Parmesan.Accounts.n.History. The old conversations are stored in XML format so they can be used to demonstrate Parmesan's XML display functionality. If you drag an XML file to Parmesan, it is opined in a new window. If you then drag the Chat CSS file stored at !Parmesan.Styles.Chat to the new window, you will see the conversation styled as it was originally, when the conversation first happened.

Parmesan vs Grapevine

I've owned Grapevine since it was first released. Unlike Parmesan it supports multiple chat networks; MSN Messenger, ICQ and IRC. I originally bought it for its MSN Messenger functionality and it introduced me to IRC, which I now use daily via another client. ICQ support was added later but I have never used it. It is clear that Grapevine offers support for more chat networks than Parmesan, but how do they compare on the common ground – as a MSN Messenger client?

Figure 4
Grapevine's interface

I'll start with the user interfaces. Grapevine's interface (see figure 4) is very bright and colourful with lots of buttons to click on. Sadly it has never been quite to my taste. I feel it tries too hard to look snazzy and ends up looking confused, sticks out like a sore thumb and generally seems a little vulgar. Some of its buttons just seem unnecessary, e.g. opening menus that are already available though the normal RISC OS style menu. I very much prefer the simple interface style of Parmesan which looks far more at home on RISC OS.

In terms of actually using the programs, they both behave pretty similarly although it is obvious that Parmesan has had a little more thought put into its interface. For example, Parmesan sorts the friends who are online and offline into alphabetical order; allowing you to find them easily. Normally I wouldn't even notice this but having been a Grapevine user I had been used to Grapevine's baffling ordering. Also, when chatting to friends, the multi-line text area of Parmesan is a far more comfortable message editing environment than Grapevine's single line text entry box. It is easier to read over a long message before sending and isn't such a rigmarole to edit multi-line messages.

Grapevine does have a few strengths to its interface that aren't matched in the first release of Parmesan. If you want to send a similar message to a previous one, for example to change part of a long URL, Grapevine allows you to press the up cursor key to fill the text area with a previously typed message. Parmesan lacks this convenient feature. Also, Grapevine chat windows take input focus when the mouse is clicked over the main content area, showing the conversation. In Parmesan, users must click in the text area to give the conversation window input focus.

Grapevine also has a useful feature that allows users to set their own display names for each of their contacts. This is a helpful feature if all your friends are in the habit of repeatedly changing their names to a variety of catch phrases and quotes, leaving you unable to keep track of who anyone actually is.

As far as supporting features of the MSN Messenger protocol go, Parmesan is, in my view, a whisker ahead. Its support for display pictures, nudges and even display of hand drawn messages is not yet matched by Grapevine. On the other hand, Grapevine supports file transfers (although my own successes with this has been limited), while Parmesan's current partial support is disabled.

In the stability, reliability and dependability stakes, it is difficult to draw any conclusions. I have had Parmesan for over a week and a half and I've been running it almost constantly, on a variety of RISC OS computers. It has never crashed or acted up on me – by failing to connect or disconnecting without telling me, etc – however I'd need a few months to be sure it's rock solid.

Grapevine has had a few ups and downs over the years with reliability. This is due, in part, to enforced updates to the MSN protocol, which locked Grapevine users out until an upgrade appeared. However, there have been other connection problems as well. Currently, I find Grapevine's reliability is at a bit of a low ebb – occasionally it fails to connect, stops telling me people are talking to me, stops sending my messages or stops updating the list of who's online or offline. These problems can be overcome by disconnecting and reconnecting once you begin to suspect something is amiss. However, the issues left me a bit nonplussed and resulted in me using Grapevine much less over the last year or so. Judging from the Grapevine mailing list and my chats with other users, these problems plague some people but other Grapevine users get along without any problems.

Conclusion

Well, I have to say I think Parmesan is awesome. It surprised me when the beta version was announced and I was very keen to test it. I am really happy with its general appearance and think the interface was designed with great skill. I like the display pictures feature and have found it stable and reliable so far. Parmesan has not been off my iconbar since I downloaded the beta and it was soon added to the "run on startup" lists on my computers. It is clear that there there are more features that can be added to Parmesan and that the interface could be improved a smidgen here and there but Parmesan is already an extremely useful and pleasant to use application. I can't wait for the next release.

Links

• Parmesan web site
• Parmesan TODO list
• Drobe's coverage of the beta release
• R-Comp's Grapevine

13 comments in forum ]]> Fri, 14 Sep 2007 21:30:00 GMT http://www.iconbar.co.uk/comments/rss/news1162.html http://www.iconbar.co.uk/comments/rss/news1162.html Hello and welcome to the Building the Dream, a new series of (regular!) articles at The Icon Bar in which I will be educating you in how to turn your programming dreams into reality. First off, let's get one thing clear - this isn't a beginner's course to programming, or a tutorial in a specific language. Hello and welcome to the Building the Dream, a new series of (regular!) articles at The Icon Bar in which I will be educating you in how to turn your programming dreams into reality. First off, let's get one thing clear - this isn't a beginner's course to programming, or a tutorial in a specific language. Instead it's the place to go once you've finished your programming tutorial and are wondering what to do next. If you have an idea for a program, but are confused about how to implement it, then this is the series for you, as I'll be covering everything from data structures and program design through to project management, optimisation, how to make sure your programs maintain the RISC OS look and feel, and even provide case studies of how certain well-known programs do their stuff.

Container data structures

Deciding which container data structures to use is often one of the most important decisions to make when designing a program. And yet they are a subject that is rarely covered properly in programming tutorials - for the simple fact that tutorials teach you how to read and write a specific programming language, rather than how to actually design a program. For example, English lessons given to a foreigner will teach them how to read and write English, but it will not teach them how to write good jokes, or compelling dramas.

Selection of the correct container data structures for use in a program can do anything from make the code a hundred times faster, to reduce memory requirements, or to make the source code cleaner (in turn making it easier to write, debug, and expand).

Typically, the most commonly used data structures are ones which are implementations of abstract data types. Whereas an abstract data type (or ADT for short) specifies what operations can be performed on a data structure (and in turn some of the ways in which the data can be processed), a data structure is the concrete implementation of that type, specifying how much memory is required and, crucially, how fast the execution of each operation will be. However for most purposes the terms ADT and data structure are interchangeable, so don't worry too much about understanding the differences between them.

Broadly speaking, container data structures are implemented using two core data types/structures - arrays and linked lists.

Arrays

I'm hoping that you already know what an array is, so I won't bother explaining it. However you may not know that a computer's RAM is essentially one big array. The memory management code in the OS then splits this array into sections, providing each program with its own area of workspace. This is an important fact to remember when writing programs, and its importance will become clearer as we go deeper into the workings of the computer in the following articles.

Since arrays are so fundamental to the construction of modern computers, it's no surprise that most programming languages have builtin support for arrays. However not all languages provide the same level of support - BBC BASIC, for example, does not allow you to resize an array once it has been created. C does allow you to resize arrays, but this can only be done to dynamically allocated arrays, and will often require you to manually update pointers (since the location in memory of the array has changed). Additionally, BBC BASIC provides range checks on array indicies, but C typically does not.

Linked lists

Linked lists can be seen as a 'dispersed array'. A typical linked list contains an ordered sequence of data elements, much like an array; but these elements are not necessarily stored in adjacent memory locations. Instead, each element in the list has a pointer to the location of the next element. A pointer to the first element of the list is also maintained - without this pointer, a program would usually have no idea where any of the elements in the list are. Some lists (called doubly-linked lists) also have pointers to the previous element, which makes some operations simpler to implement or quicker to execute. Similarly, some list implementations may keep pointers to both the first and the last element of the list, to allow the elements at either end of the list to be quickly indexed.

Because of their more complex nature, few programming languages provide direct support for linked lists. However linked lists can be implemented in almost all programming languages (either by the use of pointers or arrays).

Differences

There are several key differences between linked lists and arrays - these are surmised below:

• An array takes up a contiguous block of memory, whereas a linked list does not
• Arrays have their size dictated at creation, whereas linked lists start empty and can grow until all available memory is full
• Accessing the Nth element of an array is very quick, but with linked lists it is very slow.
• However inserting an element into the middle of an array is very slow, whereas with linked lists it is very quick (providing you have already looked up the address of the element you want to insert before/after)
• Similarly, lists can easily be split into sublists or inserted into the middle of other lists, whereas with arrays it is a bit harder (i.e. takes longer)

Very quick and very slow

A formal notation has been developed to indicate how fast algorithms and data structures are - it is known as Big O notation, and it's important that you understand it before we go any further. Big O notation is generally used to indicate how fast an operation on a data structure is, in relation to how many items are currently being stored by that structure. For example, accessing the Nth element of an array always takes O(1) time - because the number of elements in the array has no bearing upon how fast the array lookup is. But accessing the Nth element of a linked list takes (up to) O(n) time, where n is the number of elements within the list at that point in time. (Typically, lowercase n is used to indicate the number of elements within the data structure). Big O notation is usually used to indicate the maximum time the operation will take (as opposed to the average time). Common Big O denotations for data structures are O(1), O(log n), O(n), and O(n²). Depending on how large your data set is, and how frequently you wish to perform specific operations on it, the way the Big O timings affect your choice of data structure will differ. For example, you would usually want to stay away from any data structure that has O(n²) timing for an operation that you want to perform regularly, because the time taken to perform the operation can easily grow from seconds, to hours, to days, to weeks as the number of data items increases.

Abstract data types

Although I could continue this article by listing some of the more exotic data structures, I think it's more worthwhile to you if I instead list the ADTs that those data structures are implementations of. This is because you will usually first choose the ADT that meets your requirements (in terms of how the data can be manipulated), before then going on to choose the implementation that meets your timing requirements (which is where the Big O notation comes in). Or at least, it's a lot simpler to explain it this way.

Although there are many ADTs in total, I'll only be aiming to cover the core group:

List-based ADTs

These are lists, queues, and stacks. These three types are usually very similar in their implementations, which is why I've grouped them together like this. A list data type, in its most general form, allows you to insert and remove items at any point, as well as to navigate back and forth through the list, either looking for a specific item, or performing a specific operation on each item. Queues and stacks are often implemented using a list (or an array if the maximum size is known); but they restrict usage so that elements can only be inserted and removed from specific ends of the list. These restrictions allow for a faster and cleaner implementation of the data type, but at the cost of flexibility should the program's needs change in the future.

Queues are, as their name suggest, like a queue of people at a bank or a theme park - new people (elements) join at the back of the queue, but only the people (elements) at the front are removed. They are known as FIFO (First-in-first-out) data types. Similarly, stacks are often explained as being like a stack of plates - plates can only be added and removed from the top of the stack, else it falls over and someone gets shouted at by their wife. These are known as FILO (or LIFO) data types - First-in-last-out/Last-in-first-out.

The two main ways of implementing lists, queues and stacks are using either a linked list or an array. These two choices provide different tradeoffs between memory usage, insertion/removal speed, and indexing (searching for the Nth element).

Although the core specifications for most ADTs provide limited functionality (typically the minimum level required for use of the type), most implementations provide further functionality, such as checking how many items are in the data structure, checking whether a specific item is contained, or the ability to examine the next item in a stack or queue without actually removing it. It's important to bear this in mind when choosing an implementation of an ADT, as if the implementation doesn't provide the interfaces your code will require, then it's probably quite useless to you.

Maps

Maps (also known as associative arrays) are essentially a list, where each element within the list contains both a key and a value. The key part is used to access the value part, in a similar way to how an integer array index is used to access a specific item in an array. However unlike an array, the key can be of almost any data type, often one specified by the user. Most map implementations even allow keys of different data types to be used within the same map. Similarly the value can be of almost any data type, and does not usually need to the the same type for every entry in the map. Most implementations, however, place a restriction on the content of the map such that each key maps to a maximum of one value (the exception to this rule being multimaps).

There are many different implementations of maps available, each offering different execution speeds, which probably goes to explain why maps have only recently started appearing as built-in types in programming languages. For example Perl and PHP have built-in support for maps. This makes them very easy to use for certain operations, serving to increase their popularity as scripting languages. Unlike list-based ADTs, maps exist only to help you store and organise data, rather than to place restrictions on how that data is processed.

A good-quality map implementation will provide a Big O time of O(log n) or better for insertion, removal, and indexing of elements.

Sets

Sets are probably the simplest collection of objects (i.e. items of data) you can get. Typical set implementations provide three interfaces - to add an item, to remove an specific item, and to list all the items that are present within the set. Usually multiplie copies of the same item can be inserted, and usually no guarantees are made about what order the items will be listed in. Because of this there are many different ways of implementing sets, each resulting in different timing values for operations.

Sets are typically implemented in the same way as maps (using a hash table or (self-balancing) binary search tree), so you can expect performance of O(log n) or better for the common operations.

Trees

Trees are typically used within programs to increase the execution speed of operations such as searching for data or performing operations on data, while at the same time allowing for rapid insertions and removals of data items from the tree structure. The name 'tree' was chosen because, diagramatically, the organisation of data within a tree represents the organisation of branches and leaves on a tree. A tree is a series of nodes; each node has zero or more children; each child is either another node, or a leaf. Depending on implementation, either the nodes or the leaves contain the data elements of the tree. Sometimes both contain data, in which case there is usually little or no distinction between a leaf and a node.

Some tree implementations allow the tree to be split into subtrees, and for those subtrees to be glued back together again. As with splitting a list into sublists, this can be a very efficient way of changing the structure of your data, should your application require it.

Priority queues

Priority queues are queues in which each element has a priority associated with it. This priority is used to order the entries in the queue, so that the entry with the highest priority is always the one to be removed next. Because the contents of the queue needs to be kept in a sorted order, linked lists are not usually used for the implementation (because sorting a linked list can take a long time). Instead, trees or heaps are often used instead, providing much better performance.

Support in programming languages

As previously mentioned, different programming languages have different support for different data structures/types. This means that in some cases you may want to base your decision about which programming language to use around what ADTs are directly supported by that language, in terms of what ADTs you think you will be needing to use in your program.

In general, older programming languages, such as assembler, BASIC, and C, have the weakest built-in support for ADTs, so you will have to either write your own code or rely upon 3rd-party libraries (such as GLib for C - a RISC OS port of which can be found here). Newer languages, such as Perl, PHP, C++, Java and C# have better support - whether through the use of ADTs as language primitives (such as maps in Perl and PHP), or through well-defined standard implementations and interfaces that come with the compiler (such as the C++ standard template library, or the Java class library).

Conclusion

Although I haven't gone into as much detail as I'd hoped - in particular about how fast specific operations are on specific implementations of data types, or even how the implementation of each data type works - I hope I've opened your eyes a bit to the different ways that you can store and manipulating data within your programs.

Next time...

Next time I'll be talking about the RISC OS sound system - everything from understanding the terminology used to writing your own sample player.

8 comments in forum ]]> Sun, 15 Jul 2007 11:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1163.html http://www.iconbar.co.uk/comments/rss/news1163.html Hugh Robinson has contacted us to let us know that he's converted classic Acorn platformer Asylum to C, using the SDL library. With full support of original author Andy Southgate, Hugh's source code has now been released under the GPL, and is available to download from the SVN repository on the SourceForge project page. Hugh Robinson has contacted us to let us know that he's converted classic Acorn platformer Asylum to C, using the SDL library. With full support of original author Andy Southgate, Hugh's source code has now been released under the GPL, and is available to download from the SVN repository on the SourceForge project page.

Although a quick look at the source suggests to me that it's fully converted, there are still some bugs and compatability issues to sort out, so feel free to send any fixes Hugh's way if you manage to get the game running. Although the source to Asylum has been available on asylum.acornarcade.com for a few years now, this is the first known port of it to any other platform (and could potentially form the basis of a back-port to RISC OS, to produce a fully 32bit compatible version).

3 comments in forum ]]> Mon, 09 Jul 2007 20:30:00 GMT http://www.iconbar.co.uk/comments/rss/news1161.html http://www.iconbar.co.uk/comments/rss/news1161.html What a day - the Wakefield RISC OS show AND Doctor Who. It doesn't get more exciting than that. Here's my report.

Wakefield: 2007

	First steps through the door, and the first item on show is this: the A9wai1 - a widescreen, all in one RISC OS computer. Like the iMac, the hardware is built into the monitor. A wireless keyboard and mouse can be used (providing a suitable antenna is fitted inside the metal case), meaning there's only one cable required - the power lead.
	Advantage 6 also have drivers for a touchscreen version, although this wasn't on display, and the touchscreen isn't widescreen. The A9wai1 isn't a commercial product (yet), but the sort of development that Advantage Six demonstrate to potential clients and partners. The A9home is also often shown to clients as proof that their hardware designs can be successfully taken into production.
	Chris McDrobe Williams counts the number of articles he's read on the Icon Bar this year on one hand.
	Vince M Hudd of Soft Rock Software was selling WebChange for £0.00 including VAT.
	Vince stuck in Quicksand.
	Virtual Acorn running on a G5 iMac. Shiny! Although the beta is only for PowerPC processors, Aaron Timbrell promised that users who bought the beta could exchange their CD for the full Universal Binary edition, which would support both PPC and Intel processors.
	A fine demonstration of lens distortion. And Virtual Acorn.
	Virtual Risc PC again, there. The software is locked to the hardware, although software configuration and USB devices shouldn't cause any issues.
	Virtual Acorn is still in beta. I tried breaking it, but couldn't. The only issues I could see were that resizing the main window didn't cause the RISC OS desktop to resize or scroll bars to appear, so parts of the desktop became inaccessible. The preferences window was a little basic looking too. Apart from that, fine, especially in full screen mode.
	USB missile launcher on the CJE stand.
	Chris Evans demonstrating the A9 (hidden behind the blue speakers).
	Somebody had donated a Microdigital Omega to the charity stand.
	The Omega had been sold by 2pm.
	An Acorn Atom.
	A batch of new Qercus magazines arrived shortly after the show started, and John Cartmell eagerly ripped open the box. The latest issue, available to subscribers on the day, includes a review of the A9home.
	John Cartmell hides from the paparazzi.
	The Vigay.
	Obligatory Paul Vigay headshot. One day I will remember to press the red-eye button on my camera. One day.
	RISC OS Open Ltd. Sources for several RISC OS applications and components are now online. Draw and the Pinboard are items of particular interest to me. Now that the licence has been agreed, there's a lot of work to do checking the source for NDAs, project names, etc - not just in comments, but also in the source, compilation directives, and build scripts. They can release binaries for some of the components whose source isn't ready for release, but which are required to build the publically available bits. Most importantly, in addition to the mouse mats there are also coasters. Woo!
	Several people thought this said 'wait', but it doesn't. It says 'wai1', and is pronounced 'wail'.
	Giant hand, tiny box.
	Obligatory Icon Bar screenshot. Advantage Six have internet access!
	Me laughing at Drobe's live show report. Drobe reported that the A9mini was real, you see. Matt from Advantage 6 took this photo for me because he didn't want to be in it. Because the A9mini is a joke.
	The A9mini. It's a joke!
	Chris McDrobe Williams taking a photo of the A9mini. Yes, I know that he knows that it isn't real. Did I mention it's a joke?
	The RISC OS Packaging Project.
	Joel's 8 Bit Emporium. Overheard: "I bet nobody will buy this for 10p!"
	The charity stall.
	David Holden at the APDL stand.
	RISC OS SIX. Just look at all that lovely documentation!
	The huge NetSurf banner. The printer that prints this also prints doors.
	The canal, there. We didn't meet Stanley Ferry, but he seems quite a famous chap around these parts.
	Aaron Timbrell was representing both Virtual Acorn and RISCOS Ltd.
	RISC OS 6 was on sale for £49, or £99 including a Select subscription. Cheap as chips!
	Louie Smith attracts another subscriber to RISC OS Now.
	The lovely Louie Smith.
	The show floor. Although the venue is smaller than before, attendence remains high.
	No flash. Lots of blur. I like to call this an "action shot".
	Jon Ripley and Jonathan Harston like Beebs.
	Archive Magazine. Archive's editor, Paul Beverley, was one of two exhibitors wearing bow ties today...
	...the other being Martin Wuerthner. Released alongside the new version of ArtWorks is an updated rendering module which allows other applications to use the new features of ArtWorks such as transparency.
	Martin Wuerthner is also the developer for Easi/TechWriter. Forthcoming developments (possibly in time for the South-East show?) include a new style editor.
	Michael Drake, of NetSurf and Acorn Arcade fame.
	The first stable release version of NetSurf is out. The NetSurf guys were distributing mini-CDs containing the application and all the required sources and libraries to build it.
	Rob Kendrick at the NetSurf stand. Javascript was the most requested new feature - but it's a stonker, requiring significant internal changes to the way NetSurf structures and renders web pages. Rob suggested it might be possible to integrate support for some simple functions, such as submitting forms, but it's the sort of work that would have to be discarded when implementing Javascript properly. ("Properly" meaning primarily the document object model, allowing scripts to manipulate the contents of the page.)
	R-Comp had a long stand that was very difficult to photograph because customers kept getting in the way. Stupid customers. On show were a range of RiscCube machines, along with new applications like Messenger Pro 5. I'd not tried the built-in message editor until today, and it's surprisingly nice to use (I'm wary of anything which isn't StrongED or Zap). An update to Hermes is likely to include better RSS handling.
	Jack Lillingston explained how his companies and projects operate. Castle collect the revenue from RISC OS licences, and licence Iyonix Ltd to sell the Iyonix with RISC OS. They might or might not do development work on RISC OS themselves. He would be happy to see RISCOS Ltd join the party, but acknowledges that they are a separate company who do things their own way, adding value to the operating system. ROOL said that ROL's re-organisation of the RISC OS framework was an understandable approach, and it means that different implementations have been done to achieve the same end - merging these changes, if done, will require quite a lot of work, but because Castle's branch of RISC OS is publically available it should make the job much easier. Jack said that releasing the source freely for individual use, with a small licence fee for commercial use, was the most reasonable way for everyone involved.
	Matt Edgar from Advantage Six. He thinks there's still a market for RISC OS. Although some people may come to them with an idea specifically to use RISC OS (which they'd consider with a viable business case), there are also clients who simply need their requirements fulfilling without a particular OS in mind - and if RISC OS is the right solution they'll use it. Although not all the work Advantage Six do is RISC OS related, the RISC OS stuff is fun... and that's what life is about!
	Simply messing about in boats.
	It's almost a cliche to say so, but this year's Wakefield had a real positive vibe. There's such a contrast between the whinging, arguing and trolling on the internet and friendly, charming conversations you have at shows like this. There was no doom and gloom; quite the reverse, and with a recognition of what RISC OS now means to people. The change in venue is an example - there's no longer the need or opportunity to pretend it's an event to rival Earl's Court or - more importantly - to feel disappointed that it isn't. As a chance to put names to faces, meet distant friends, and see the technology you've believed in for decades still being applied in interesting and successful ways, the show couldn't be anything other than a success.

Doctor Who: 42

I liked it when the sun took over the Doctor and he started shouting with his eyes on fire.

16 comments in forum ]]> Sat, 19 May 2007 20:00:00 GMT http://www.iconbar.co.uk/comments/rss/news1160.html http://www.iconbar.co.uk/comments/rss/news1160.html "VirtualAcorn are delighted to announce that as part of our beta product testing scheme a limited number of VirtualRPC-AdjustSA products suitable for use on G4 and G5 PowerPC Apple Macintosh computers are now available." "VirtualAcorn are delighted to announce that as part of our beta product testing scheme a limited number of VirtualRPC-AdjustSA products suitable for use on G4 and G5 PowerPC Apple Macintosh computers are now available."

http://www.virtualacorn.co.uk/products/vrpcadsamac.htm

19 comments in forum ]]> Thu, 17 May 2007 11:22:00 GMT http://www.iconbar.co.uk/comments/rss/news1159.html http://www.iconbar.co.uk/comments/rss/news1159.html It's seemingly mostly doom and gloom these days in the increasingly small world of RISC OS, with the glaring deficiencies in the platform still unresolved and the latest releases from the OS developers having received an underwhelming response. However, it's always easier to concentrate on the dark side, and possible to overlook the fact that there's nice stuff happening too. It's seemingly mostly doom and gloom these days in the increasingly small world of RISC OS, with the glaring deficiencies in the platform still unresolved and the latest releases from the OS developers having received an underwhelming response. However, it's always easier to concentrate on the dark side, and possible to overlook the fact that there's nice stuff happening too. So I thought I'd highlight a few of those developments here in this article.

Now, before anything else, I should make clear that this is *not* an attempt at advocacy: I'm not trying to argue that RISC OS is in a good state, and certainly not that users of other OSes should switch over (or back). All I'm aiming to do is talk about some areas where RISC OS seems to me to be getting a bit better. And there are more of these than might be obvious, at least if reading Usenet is your guide to the health of the platform.

File formats

First off, reading and editing of industry-standard document files is much better now than it was in Acorn's heyday. Take PDFs, one of the most important standards in both printing and on the internet. A few years ago the options for manipulating complex PDF files were limited. Now, most of the PDFs I download are readable in RISC OS. I use either !PDF, a free viewer which is quick on my A9 and generally gives good results. This won't read everything out there, but if it fails, I fire up the excellent GView from Martin Wuerthner, which is also free (but if you find it useful, please donate to support this important app). This has coped with all sorts of things I've thrown at it. It is slow, to be sure, but certainly not ridiculously so on modern hardware.

In addition to this, there's RiScript, a commercial application for reading and editing PDFs, and increasingly comprehensive PDF export from EasiWriter and ArtWorks (the latter is due for further improvement with the imminent release of 2.7). Now, I daresay there are lots of things one can't do with PDFs on RISC OS, and I wouldn't want to suggest matters are perfect. But the point is that this situation is much advanced from where used to be, and promises to develop further.

PDFs are one example of better document reading, but there are others. Word file import and export is now very nicely done in EasiWriter, which benefits from constant development. The freeware application ViewXLS can read Excel files, and the commercial spreadsheet Schema is being augmented to cope with these better too. Though large holes remain, the RISC OS community seems to have grasped the idea that support for industry-standard file formats is a good idea: we can cautiously hope that this trend will continue, and some gaps (such as PowerPoint) will be filled in due course.

Printing

Right back in Acorn's heyday, printing was a bit of a pain. Supporting third-party hardware was tough even when RISC OS computers were relatively common in the UK, and once the brand disappeared and resources dwindled, the range of supported printers got smaller. But now, thanks to the magic of the open-source community (and, more importantly for us, the skills of Martin Wuerthner), things look a lot brighter. Using Gutenprint, a port of the printing system used by many Linux set-ups, dozens of modern printers are now supported, with the expectation that many more will be added as the legions of spotty-faced bedroom developers around the world add drivers and documentation. It's not exactly a perfect system (again, it's slow on current hardware), but just think where we'd be without it. It's an excellent initiative, deserving of the support of every RISC OS user.

And for those who network their machine to a Windows PC, there's also the ingenious UniPrint from R-Comp, which (among other things) allows printing using the PC's drivers instead of the increasingly out-of-date Acorn set. Of course, this does depend on having access to a PC in the first place, but it adds a way of printing where no other may exist. Not everyone's cup of tea, I'm sure, but there's no doubt many users find it invaluable.

Application development

Again, it often seems to pass unnoticed how much quiet development there is going on in the RISC OS scene. Most attention is focussed on what we lack: a stable Javascript browser, a full-featured media player, etc. But although it's quite understandable (and right) to mourn the absence of these important things, there is other stuff going on. For example, musicians in the past couple of weeks have seen updated MIDI modules, a new 32bit Rhapsody commercial application, the 32bitting of the audio editor StudioSound, and the continued availability of superb software such as SamplEd and MelIDI. This is a pretty good haul for a platform at death's door!

Let's also celebrate other gradual improvements which seem to get drowned out from time to time: a major update to APDL's Ancestor family tree program, continued Messenger Pro development, ongoing StrongED work (with an update to version 4.68 hopefully around the corner), DigitalCD improvements, and work on CDVDBurn. These are important applications, meeting users' needs, gradually adding features and filling in gaps. And besides these there are a host of smaller, usually free utilities which get released all the time: a trawl of csa.announce over the past few days brings up half a dozen small apps, none earth-shaking in themselves, but all in response to a genuine niche requirement.

Most of the above are commercial projects, or RISC OS-exclusive open-source applications. But perhaps the most exciting and hopeful work being done at the moment is in the field of cross-platform open-source. Although not obvious to everyone, over the past few years an enormous amount of effort has been devoted to getting the GNU compiler collection (GCC) to work on RISC OS. Recently, it's even been possible to compile RISC OS applications on other platforms using GCCSDK. This is a truly momentous accomplishment, which many other minority platforms must envy. If it doesn't seem so significant to you, consider that without GCC and its related projects we would have no NetSurf, no Firefox, no Gutenprint, no Unix ports, and no real prospect of getting the media player we need. The developers of much of this get little thanks (and occasionally criticism) – I often wish there was more appreciation and understanding of how important this work is.

OK, I'll stop now

By this point, even the most ardent advocate of RISC OS is likely to be getting a little sceptical: surely things aren't as rosy as the above picture suggests? And of course they aren't: our OS has a lot of major challenges, and I truly have no idea whether we'll even be around this time next year. But I make no apology for celebrating some of the hard work that is producing fruit – it's good for the soul to recognise what is functioning well, even if it contributes in only a small way. I wouldn't blame anyone for moving on to other OSes with more features and software, but I do think there is some fun to be had with RISC OS, and we're not quite dead yet.

5 comments in forum ]]> Mon, 07 May 2007 13:00:00 GMT