Downloading Data from the Internet

There are a thousand reasons why you might want your application to get information from the Internet. Perhaps you want to download today’s Dilbert cartoon, or get specific data for the user, or serve adverts, or  connect to a web interface. Maybe you want to implement a new protocol, like FTP or Jabber. In this tutorial we’re going to show you how to get data over HTTP.

The Connection Object Model

MoSync provides a wide range of connection objects and interfaces. There are three levels of abstraction for Internet connections, and you can write code against any of them:

• Using the maConnect() API method
• Using the MAUtil::Connection class
• Using the Downloader classes

Using maConnect() API Method in C/C++

At the lowest level, you can use functions in the MoSync API directly. You can use the method maConnect() to open a socket to connect to any port on the Internet. You can implement protocols which aren’t supported in MoSync on a higher level (like HTTP and HTTPS) by formatting a URL like this:

socket://<server name>[:<port>]<parameters>

so you can create your own protocol support for IMAP and POP3 to make an email client, and the documentation come with the basics of an FTP client.

You can use this method to connect to Bluetooth devices as well, with the format

btspp://<address>

We are going to concentrate on HTTP connections in this tutorial though, and although the API provides some HTTP specific methods, we’re going to focus on the download components in MAUtil.

Using the MAUtil::Connection Class in C++

The second level of abstraction is with MAUtil::Connection objects. These are C++ objects and are much more suitable for use in a Moblet application.

• MAUtil::Connection allows you direct access to the send and receive streams.
• MAUtil::HttpConnection builds HTTP 1.0 connections.

With HttpConnection objects you can access the HTTP 1.0 protocol so you can set HTTP headers before you send, and create new wrapper classes for implementing specific HTTP functions. For instance, you can use  HttpConnection to implement an online authentication system, which you can then reuse.

Connections send messages to ConnectionListeners. When you are working with either a Connection or an HttpConnection, you will almost certainly need to create a class which inherits from ConnectionListener or HttpConnectionListener, and implements its methods. We find that very often this class is "my screen", which we define as:

class MyScreen : public Screen, ConnectionListener

and then implement the methods:

void connectFinished(Connection* conn, int result);
void connRecvFinished(Connection* conn, int result);
void connWriteFinished(Connection* conn, int result);
void connReadFinished(Connection* conn, int result);

These methods are your opportunity to respond to data coming back, and you can do whatever you want in there.

Using the Downloader Classes in C++

At the highest level are the Downloader classes. These are very high level classes that wrap much of the mechanics of getting data over HTTP for you. These are the classes which we use most often, and if you just want to download some XML or a picture then these are the classes you should use too.

The Downloader class itself wraps HttpConnectionListener, and it is very simple to use. You simply tell it the URL you want to get data from and give it an MAHandle to load the data into.

If you want to know when your download has completed, you have to implement a listener too. You do this by inheriting from DownloadListener and implementing the interface. There is an example of this in the next section of the tutorial.

A second downloader is the BuffDownloader. This is rather more specialised, and instead of downloading to a placeholder, it downloads to the heap.

There are two downloaders at an even higher level: AudioDownloader and ImageDownloader. These are both wrappers for Downloader, but create the appropriate resources in memory for handling these items. More on these later.

Finally, you can create your own custom Downloader. You can inherit from Downloader, and perform some of your own downloading tasks, just as you can implement Connection to manage your own application protocol.

Downloading Data from the Web

In all probability, what you actually want to do is download some data from the web. The easiest way to do this is to use the Downloader class and to create a DownloadListener to go with it. To do this we can make my Screen which requests the download from the DownloadListener. To do this in a reusable way, we create a DownloadScreen class.

DownloadScreen.h

#ifndef _DOWNLOADSCREEN_H_
#define _DOWNLOADSCREEN_H_
#include <MAUI/Screen.h>
#include <MAUI/Layout.h>
#include <MAUtil/Downloader.h>
#include "IScreenCoordinator.h"
#include "..\Utilities/Util.h"
#include "..\Utilities\IDownloadScreenListener.h"
#include "..\Widgets\SoftKeyBar.h"
using namespace MAUI;
using namespace MAUtil;
class DownloadScreen : public Screen, public DownloadListener, public ISoftKeyBarListener
{
public:
    DownloadScreen(Screen* previous, IScreenCoordinator* mainScreen);
    ~DownloadScreen();
    void download(const char* url, const char* storeName);
    void keyPressEvent(int keyCode);
    void setDownloadListener(IDownloadScreenListener* dll);
    void notifyProgress(Downloader *dl, int downloadedBytes, int totalBytes);
    void finishedDownloading(Downloader *dl, MAHandle data);
    void error(Downloader* dl, int code);
    void downloadCancelled(Downloader* dl);
    void softKeySelected(int buttonID);
private:
    Screen* previous;
    IScreenCoordinator* mainScreen;
    Layout* layout;
    Image* image;
    ListBox* listBox;
    Downloader* dl;
    IDownloadScreenListener* listener;
    MAHandle _store;
};
#endif //_DOWNLOADSCREEN_H_

This code will build my screen, manage the downloads, and inform its own type of Listener (IDownloadScreenListener) when downloads are complete. (It doesn’t inform the IDownloadScreenListener of anything other than successful completion, as the user interaction for a failed download can be handled by this screen.)

IDownloadListener.h

#ifndef _IDOWNLOADSCREENLISTENER_H_
#define _IDOWNLOADSCREENLISTENER_H_
//Interface
class IDownloadScreenListener
{
public:
    virtual void downloadComplete();
};
#endif //_IDOWNLOADSCREENLISTENER_H_

DownloadScreen.h is implemented in DownloadScreen.cpp:

#include  "DownloadScreen.h"
//Handles a download
const char* store;

DownloadScreen::DownloadScreen(Screen* previous, IScreenCoordinator* mainScreen)
 : previous(previous), mainScreen(mainScreen)
{
    dl = new Downloader();
    dl->addDownloadListener(this);

    image = (Image*)createMainLayout(BLANK, BACK_BUTTON, this);
    layout = (Layout*) image->getChildren()[0];
    listBox = (ListBox*) layout->getChildren()[1];
    Label* l = createLabel("Starting download", 32);
    listBox->add(l);
    this->setMain(image);
}

DownloadScreen::~DownloadScreen(void)
{
    if(dl->isDownloading())
    dl->cancelDownloading();     
    delete dl;
}

void DownloadScreen::setDownloadListener(IDownloadScreenListener* dll)
{
    listener = dll;
}

void DownloadScreen::download(const char *url, const char* storeName)
{
    if(dl->isDownloading())
    {
        //lprintfln("Busy.");
        //dl->cancelDownloading();
    }
    else
    {
        lprintfln("Downloading %s", url);
        store = storeName;
        dl->beginDownloading(url);
    }
}

void DownloadScreen::downloadCancelled(Downloader *dl)
{
    //lprintfln("Cancelled");
    listBox->add(createLabel("Download has been cancelled"));
}

void DownloadScreen::error(Downloader *dl, int code)
{
    lprintfln("Error: %d", code);
    listBox->add(createLabel("Sorry, an error has occured"));
}

void DownloadScreen::finishedDownloading(Downloader *dl, MAHandle data)
{
    //Save the store
    MAHandle h = maOpenStore(store, MAS_CREATE_IF_NECESSARY);
    maWriteStore(h, data);
    maCloseStore(h, 0);

    lprintfln("Finished download");
    if(listener != NULL)
    {
        //lprintfln("Calling listener");
        listener->downloadComplete();
    }

    //lprintfln("Showing calling screen");
    previous->show();
}

void DownloadScreen::notifyProgress(Downloader *dl, int downloadedBytes, int totalBytes)
{
    Label* l = (Label*)listBox->getChildren()[0];
    char* cap = new char[255];
    sprintf(cap, "Downloaded %d of %d bytes", downloadedBytes, totalBytes);
    l->setCaption(cap);
    delete[] cap;
    //lprintfln("Downloaded %d of %d bytes", downloadedBytes, totalBytes);
}

void DownloadScreen::softKeySelected(int buttonID)
{
    switch(buttonID)
    {
    case SoftKeyBar::LSK:
        break;
    case SoftKeyBar::FIRE:
        break;
    case SoftKeyBar::RSK:
        previous->show();
        break;
    }
}

void DownloadScreen::keyPressEvent(int keyCode)
{
    switch(keyCode)
    {
    case MAK_2:
    case MAK_4:
    case MAK_UP:
        listBox->selectPreviousItem();
        break;
    case MAK_6:
    case MAK_8:
    case MAK_DOWN:
        listBox->selectNextItem();
        break;
    case MAK_0:
        if(dl->isDownloading())
        {
            dl->cancelDownloading();
        }
    }
}

This download screen can be used from many different screen.  For instance, if you've got a screen which shows a calendar where the user can drill into different days to see what their schedule is for that day.  This screen can download a package of data from the Internet of events created for the user.  When you want to download an update, my EventScreen calls DownloadScreen and passes it the URL it wants to download.  This DownloadScreen handles all of the mechanics.

The DownloadScreen has inside it a Downloader object, and it implements DownloadListener.   It creates a Downloader when it is constructed, and deletes it when it is destroyed.  When the DownloadScreen’s download method is called, it passes the URL to the Downloader and starts downloading.

As the Downloader raises events to the DownloadScreen through the DownloadListener interface, the screen can keep the user informed about their download.  When it has finished, it informs the calling screen that it has completed, so that screen can carry on.  

To use it, call it from EventsScreen.cpp.  This is a partial listing.

#include "EventsScreen.h"
#include <mastring.h>
EventsScreen::EventsScreen(Screen* previous, MainScreen* mainScreen) 
: previous(previous), mainScreen(mainScreen)
{
	image =(Image*) createMainLayout(BLANK, BACK_BUTTON, this);
	layout = (Layout*) image->getChildren()[0];
	listBox = (ListBox*) layout->getChildren()[1];
	this->setMain(image);
	urlptr = NULL;
	dlscreen = new DownloadScreen(this, mainScreen);
	dlscreen->setDownloadListener(this);
}
EventsScreen::~EventsScreen()
{
	if(urlptr != NULL)
	delete [] urlptr;
	delete dlscreen;
}
void EventsScreen::downloadComplete()
{
	listBox->getChildren().clear();
	createEventScreen();
}
void EventsScreen::keyPressEvent(int keyCode)
{
	switch(keyCode)
	{
	case MAK_2:
	case MAK_4:
	case MAK_UP:
		listBox->selectPreviousItem();
		break;
	case MAK_6:
	case MAK_8:
	case MAK_DOWN:
		listBox->selectNextItem();
		break;
	case MAK_0:
		dlscreen->download(formatUrl(), EVENTSTORAGE);
		dlscreen->show();
		break;
	}
}

When the user presses 0 on their keypad, it tells the DownloadScreen the URL to download from, and the name of the store to write the data to.  More on that later.

When the data has been completely downloaded, the EventsScreen::downloadComplete() method is called, and the screen can repopulate itself from the downloaded data.

Receiving Events

In the above example, the DownloadScreen class implements DownloadListener.  In particular, it implemented these functions:

void notifyProgress(Downloader *dl, int downloadedBytes, int totalBytes);
void finishedDownloading(Downloader *dl, MAHandle data);
void error(Downloader* dl, int code);
void downloadCancelled(Downloader* dl);

By doing this, the download screen is informed about what is happening to the downloader.  The implementation code for these is here:

void DownloadScreen::downloadCancelled(Downloader *dl)
{
	//lprintfln("Cancelled");
	listBox->add(createLabel("Download has been cancelled"));
}
void DownloadScreen::error(Downloader *dl, int code)
{
	lprintfln("Error: %d", code);
	listBox->add(createLabel("Sorry, an error has occured"));
}
void DownloadScreen::finishedDownloading(Downloader *dl, MAHandle data)
{
	//Save the store
	MAHandle h = maOpenStore(store, MAS_CREATE_IF_NECESSARY);
	maWriteStore(h, data);
	maCloseStore(h, 0);
	lprintfln("Finished download");
	if(listener != NULL)
	{
		//lprintfln("Calling listener");
		listener->downloadComplete();
	}
	//lprintfln("Showing calling screen");
	previous->show();
}
void DownloadScreen::notifyProgress(Downloader *dl, int downloadedBytes, int totalBytes)
{
	Label* l = (Label*)listBox->getChildren()[0];
	char* cap = new char[255];
	sprintf(cap, "Downloaded %d of %d bytes", downloadedBytes, totalBytes);
	l->setCaption(cap);
	delete[] cap;
	//lprintfln("Downloaded %d of %d bytes", downloadedBytes, totalBytes);
}

When the Downloader tells the calling screen of an error, or that the download has been cancelled, or that it has downloaded some of the data, then you can tell the user by updating the screen.  When the download has finished, you can call the IDownloadScreenListener, and show the previous screen.