Custom Downloaders

There are obvious occasions when you want do download data from the Internet for your games or application.  There are even obvious occasions when you want to download images and sounds using the ImageDownloader and AudioDownloader.  What might appear less obvious is the use of a Custom Downloader.

Custom Downloaders

Custom Downloaders are more useful than you might think.  These are classes where you have implemented ConnectionListener, but got it to do something with the data before consuming it.

A great, if lengthy, example is the mapping component below.  This is a custom widget (not part of the MoSync MAP SDK) to make scrollable, zoomable maps using the CloudMade map tiles.  Initially, this used an ImageDownloader to get each and every tile separately from CloudMade.  This was very slow, not because the data connection was slow, but because the HTTP latency was so great.  It needed to negotiate a connection to the server 36 times to build a screen.

Instead, this connects to a small web application which will take requests for an indefinite number of tiles at once.  It will go and get those tiles from CloudMade, cache them in memory for quick retrieval later, and return them to me in a single download.  The application can then rip the tiles out of the stream and pass them to the map for display.

This has been implemented as a CustomDownloader.  It allows for additional logic to be wrapped in the download mechanism, so other classes do not need knowledge of the connection to get the correct data.

MapTileDownloader.h

#ifndef _MAPTILEDOWNLOADER_H_
#define _MAPTILEDOWNLOADER_H_
#include <MAUtil/Connection.h>
#include "MapTileDownloadListener.h"
using namespace MAUtil;
using namespace DatiloMapping;
namespace DatiloMapping
{
	class MapTileDownloader : public ConnectionListener
	{
	public:
		MapTileDownloader();
		virtual ~MapTileDownloader();
		void startDownloading(const char* url);
		void cancelDownloading();
		void removeListener(MapTileDownloadListener* listener);
		void addDownloadListener(MapTileDownloadListener* listener);
		void connectFinished(Connection* conn, int result);
		void connRecvFinished(Connection* conn, int result);
		void connReadFinished(Connection* conn, int result);
		void connWriteFinished(Connection* conn, int result);
		bool isDownloading();
	private:
		Vector<MapTileDownloadListener*> downloadListeners;
		Connection* mapConn;
		bool activeDownload;
		void throwTile();
		void processDownload();
		void iterateTiles(int next);
		void readKeyLength();
		void readKey();
		void readImageLength();
		void readImage();
		int dlStage;
		ushort tileCount;
		int keyLength;
		int downloadedBytes;
		int nextTile;
		bool fetchComplete;
		bool connectionOpen;
		int tileSize;
		byte kl; //key length
		char* keyBuffer;
		MAHandle h;
	};
};
#endif //_MAPTILEDOWNLOADER_H_

MapTileDownloader.cpp

#include  "MapTileDownloader.h"
#include <conprint.h>
#include <madmath.h>
MapTileDownloader::MapTileDownloader()
{
	h = maCreatePlaceholder();
	keyBuffer = new char[10];
	mapConn = new Connection(this);
	activeDownload = false;
}
MapTileDownloader::~MapTileDownloader()
{
	maDestroyObject(h);
	delete mapConn;
	delete [] keyBuffer;
}
//Start a download
void MapTileDownloader::startDownloading(const char* url)
{
	if(mapConn->isOpen())
	mapConn->close();
	activeDownload = true;
	int res = mapConn->connect(url);
	if(res < 0)
	return;
}
void MapTileDownloader::processDownload()
{
	dlStage = 0;
	nextTile = 0;
	fetchComplete = false;
	mapConn->read(&tileCount, 2); //Read exactly two bytes
}
void MapTileDownloader::iterateTiles(int next)
{
	nextTile = next;
	//Read through each tile
	if(nextTile < tileCount)
	{
		//Reset the buffers
		memset(keyBuffer, 0, 10);
		readKeyLength();
	}
	else
	{
		mapConn->close();
		activeDownload = false;
		Vector_each(class MapTileDownloadListener*, itr, downloadListeners)
		{
			(*itr)->complete();
		}
	}
}
void MapTileDownloader::readKeyLength()
{
	dlStage = 1;
	mapConn->read(&kl, 1); //Read exactly one byte
}
void MapTileDownloader::readKey()
{
	dlStage = 2;
	mapConn->read(keyBuffer, kl);
}
void MapTileDownloader::readImageLength()
{
	dlStage = 3;
	mapConn->read(&tileSize, 4);
}
void MapTileDownloader::readImage()
{
	dlStage = 4;
	//Remove any existing obejct
	maDestroyObject(h);
	maCreateData(h, tileSize);
	mapConn->readToData(h, 0, tileSize);
}
//Returns a tile to the listener
void MapTileDownloader::throwTile()
{
	Vector_each(class MapTileDownloadListener*, itr, downloadListeners)
	{
		(*itr)->finishedDownloading(keyBuffer, h, tileSize);
	}
	maDestroyObject(h);
	//Get the next tile
	iterateTiles((nextTile + 1) % 36);
}
//Handle add/remove of listeners
void MapTileDownloader::removeListener(MapTileDownloadListener* listener)
{
	Vector_each(class MapTileDownloadListener*, itr, downloadListeners)
	{
		if((*itr) == listener)
		{
			downloadListeners.remove(itr);
			break;
		}
	}
}
void MapTileDownloader::addDownloadListener(MapTileDownloadListener* listener)
{
	downloadListeners.add(listener);
}
//ConnectionListener events
void MapTileDownloader::connectFinished(Connection* conn, int result)
{
	processDownload();
}
void MapTileDownloader::connRecvFinished(Connection* conn, int result)
{
}
void MapTileDownloader::connReadFinished(Connection* conn, int result)
{
	fetchComplete = true;
	switch(dlStage)
	{
	case 0:
		iterateTiles(0);
		break;
	case 1:
		readKey();
		break;
	case 2:
		readImageLength();
		break;
	case 3:
		readImage();
		break;
	case 4:
		throwTile();
		break;
	}
}
void MapTileDownloader::connWriteFinished(Connection* conn, int result)
{
}
void MapTileDownloader::cancelDownloading()
{
	if(mapConn->isOpen())
	{
		mapConn->close();
	}
	activeDownload = false;
}
bool MapTileDownloader::isDownloading()
{
	return activeDownload;
}

This gets a stream of tiles.  It reads the first two bytes of the stream to get the number of tiles in the stream, and then for each tile it reads the key so the map will know where this image goes.  It then reads the length of the image, and then the image data itself.  As it gets each tile, it throws it to a listener which can pass it to the map, so it is drawing tiles as they come down.  By implementing a custom listener like this, you will get a speed improvement of ten-fold!