CETIS Widget Bash

On returning home from a productive and enjoyable two days in Bolton I’m rather pleased to see that our PC Availability widget displays the free PCs in the MMU drop-ins in a different order now I’m in South Manchester. Our widget became location-aware at 13:47 GMT on March 24, and popular culture would counsel against pulling the plug, so instead, I’ll try and share how we moved things on from the version we described in our previous post that is currently running in WordPress to the right of this article.

On the first day of the CETIS event, wookie champion and JISC OSS Watch Service Manager Ross Gardler described how widgets could be enhanced using open source javascript libraries, such as geo-location-javascript. While Ross was presenting I added a reference to the geo-location-javascript library to the html and some sample geo code to the pc.js file in our PC Availability widget and found that most browsers would disclose latitude and longitude after checking first with the user.

[codesyntax lang=”javascript” title=”Geo-location code added to pc.js”]

// Updated the Controller.init function
init:function() {
	if(geo_position_js.init()){ 	
           geo_position_js.getCurrentPosition(Controller.success_callback,Controller.error_callback,{enableHighAccuracy:true});
	}
	else {
		Controller.update();
	}
}

...
// Added Controller.success_callback
success_callback:function(p) {
		alert('latitude='+p.coords.latitude.toFixed(2)+'&longitude='+p.coords.longitude.toFixed(2));
		Controller.update();
}

//Added Controller.error_callback
error_callback:function(p) {
	alert('error='+p.code);
}

[/codesyntax]

Having found that I could get the latitude and longitude of a mobile device, I was keen to see if these values could be used to find the nearest available drop-in PCs. Whilst the processing could be done client side, I decided it would be more flexible for the future if our PC Availability web-service were able to receive latitude and longitude as query parameters and order its results based on proximity.

We already had geo-location data for our drop-in facilities from our work with oMbiel’s CampusM mobile app, so Kieron kindly extended the table that holds the list of drop-in facilities (an MS SQL Server known as stu_services) to include two extra fields: lat and long. I then needed to modify the C# code for our web-service to:

  1. Modify the Windows Communication Framework web-service definition to take latitude and longitude params
  2. Set the default sort order as alphabetic by location
  3. Test if the service had been called with valid latitude and longitude values, and (if it had) set the sort value based on a pythagoras calculation (which I know is not as accurate as the Haversine formula, but should be adequate for our purpose)
  4. Modify the SQL query to incorporate the sort criteria

The C# for step 1 was:

[codesyntax lang=”csharp”]

        [OperationContract, WebGet(UriTemplate = "getPcAvailability?latitude={latitude}&longitude={longitude}", ResponseFormat = WebMessageFormat.Xml)]
        PcAvailability getPcAvailability(string latitude, string longitude);

[/codesyntax]

The C# code I used within the getPcAvailability method for steps 2 and 3 was:
[codesyntax lang=”csharp”]

double dblLatitude = 0;
double dblLongitude = 0;
string sort = ", location ";

if (Double.TryParse(latitude, out dblLatitude) && Double.TryParse(longitude, out dblLongitude))
{
   sort = ", SQRT(POWER(CAST(longitude AS REAL) - CAST('" + longitude + "' AS REAL),2) + " +
            " POWER(CAST(latitude AS REAL) - CAST('" + latitude + "' AS REAL),2)) ";
}

[/codesyntax]

And the modified SQL for Step 4 was:

[codesyntax lang=”csharp”]

                SqlCommand myCommand = new SqlCommand("" +
                "SELECT location " +
                ", 	rid " +
                ", 	info " +
                ",  	latitude " +
                ",  	longitude " + sort +
                ", 	SUM(free) as 'free' " +
                ", 	SUM(pool) as 'pool' " +
                "FROM " +
                "( " +
                "SELECT usage.rid as rid " +
                ",	stu_services.location as location " +
                ",	stu_services.info as info " +
                ",  	stu_services.lat as latitude " +
                ",  	stu_services.long as longitude " +
                ", 	count(*) as 'free' " +
                ", 	0 as 'pool' " +
                "FROM 	usage " +
                "INNER JOIN stu_services " +
                "ON usage.rid = stu_services.rid " +
                "WHERE InUse='NO' " +
                "GROUP BY usage.rid " +
                ",	stu_services.location " +
                ",	stu_services.info " +
                ",  	stu_services.lat " +
                ",  	stu_services.long " +
                "UNION " +
                "SELECT usage.rid as rid " +
                ",	stu_services.location as location " +
                ",	stu_services.info as info " +
                ",  	stu_services.lat as latitude " +
                ",  	stu_services.long as longitude " +
                ", 	0 as 'free' " +
                ", 	count(*) as 'pool' " +
                "FROM 	usage " +
                "INNER JOIN stu_services " +
                "ON usage.rid = stu_services.rid " +
                "GROUP BY usage.rid " +
                ",	stu_services.location " +
                ",	stu_services.info " +
                ",  	stu_services.lat " +
                ",  	stu_services.long " +
               ") status " +
                "GROUP BY location " +
                ",	rid " +
                ",	info " +
                ",  	latitude " +
                ",  	longitude " + sort +
                "ORDER BY " +
                " 6,1 " +
                "", myConnection);

[/codesyntax]

… and it worked (on our test box at least)!
http://testapis.ad.mmu.ac.uk/icts/Service1.svc/getPcAvailability?latitude=53.47067&longitude=-2.238943

Now to modify the pc.js file of our widget to append the latitude and longitude query string parameters to the web-service URL if the device can provide them. After experiencing the benefits of the Firebug FireFox add-in as a javascript debugger, I eventually ended up with a re-worked pc.js file that worked when zipped as a widget and deployed to wookie:

[codesyntax lang=”javascript” title=”Our location-aware pc.js widget file”]

var Controller = {

	coords:"",

	init:function() {
		if(geo_position_js.init()){
			geo_position_js.getCurrentPosition(Controller.success_callback,Controller.error_callback,{enableHighAccuracy:true});
		}
		else{
			Controller.update();
		}
	},

	update:function() {

		var loc = Widget.proxify("http://testapis.ad.mmu.ac.uk/icts/Service1.svc/getPcAvailability" + Controller.coords);

		$.ajax({
		        type: "GET",
			url: loc,
			dataType: "xml",
			timeout: 1000,
			complete: Controller.parseResponse
		});
	},

	parseResponse:function(response) {

		var rooms = $("#rooms-listview");
		rooms.empty();
		$(response.responseXML).find("room").each(function () {
			rooms.append($("<li/>").text($(this).attr("location") + ": " 
			+ $(this).attr("free") + "/" 
			+ $(this).attr("seats") + " free"));
		});
		rooms.listview("refresh");
	},

	success_callback:function(p) {
		Controller.coords = '?latitude='+p.coords.latitude.toFixed(2)+'&longitude='+p.coords.longitude.toFixed(2);
		Controller.update();
	},

	error_callback:function(p) {
	}
}

[/codesyntax]

This code needs tidying – the web-service URL should be read from a properties file, etc – but hopefully this quick post will help maintain the excellent spirit of community development that everyone enjoyed at the CETIS Widget Bash. Thanks to Sheila, Li, Sarah, Ross and Scott for organizing and to all who attended for making it such a valuable event.

Running with Wookie

When we first looked into getting involved with the Widget revolution we wanted scalable widgets that could enhance the student learning experience and be deployed on a range of different platforms.

Walking with Wookie

Er… thought you said Running with…

Well, when we first looked into getting involved with the Widget revolution we wanted scalable widgets that could enhance the student learning experience and be deployed on a range of different platforms. Indeed, we still do. We are in the process of deploying Moodle, hosted by the University of London Computing Centre (ULCC), as our institutional VLE, and were attracted to the potential of widgets as a way to enhance the VLE and be available on mobile devices. We realized that if widgets were to be part of our core offering, we’d need a widget server that could handle multi-thousands of hits in a short time interval, so began exploring Wookie with that in mind.

I proceeded to put a test platform together on our existing webserver and opted for the Wookie install that utilises Tomcat and MySQL so that we could potentially load-test the platform at a similar level to our existing web platforms.

I have to say the process wasn’t attractive or easy. “Dependency hell” took over fairly soon with trying to determine which JDK to run, out of the three that lived on our system (an Ubuntu derivative), making sure we had the right JDBC and that it could talk to MySQL (also having to be set up correctly). That said, much of this is really dependent on the platform you wish to run Wookie on, the variant or distro and so on. Having revisited the Wookie trunk in recent weeks I can say great strides have been made in only a year in making Wookie easier to install. You still really have to beware of the various Java JDK’s out there – stick with the Oracle one would seem to be safe advice.

The development team are fantastic, frenetic and focused – I cannot recall too many other software / platform development projects that I have been involved with that have released so many upgrades, patches and fixes over such a short period. Wookie itself is still a project under Apache incubation, in theory a kind of Beta state. This means you need to expect a certain amount of work to get the software going on a system of your own. In my struggles I was able to visit Paul and Scott (and also Scott’s interesting personal page ) in Bolton and get a couple of errors sorted out to enable our install to function. This actually fed back into the process at the time – a script hadn’t performed as it should, was corrected, and the nice new working version made its way back into the build. Just goes to show how fast communities can rectify problems!

Result – a working copy of Wookie on a server base.

Problems with the Server-based install

At some point I opted to patch our system (as you should, once everyone else has done it and found the bits that break!), ANT – an essential component climbed a couple of revision levels and promptly broke the Wookie install script. This is kind of expected behaviour when working at the beta end of things, but it came at a time when we needed to start working on our widgets: not good! The team have been working to sort out all sorts of similar issues throughout the year, but with increasing interest in developing our own widgets to deploy into our enhanced VLE platform (check out Mark Stubbs’ recent post Writing Widgets) we revisited a server-side copy of Wookie. ANT is still not working – what to do?

Running with Wookie – use the Wookie standalone install

[pullquote]Useful  links

[/pullquote]

 

Given that our project really needs to focus on the development of Widgets, a rethink was needed on applying efforts towards writing Widgets, and not messing with Wookie. Whilst visiting the developers, Scott and Paul, in Bolton about some issues regarding our server-based installation I noticed that the folks there are essentially all developing against local standalone copies of Wookie.

I decided to give it a try. Simply put, it’s the fastest and easiest way to get Wookie going. I have no stats to tell you how robust the standalone copy is when operated within a typical server situation, but thus far, for our tests, it has remained up and viable.

So – what are the crucial benefits of using the standalone copy?

Firstly, ease of install. It can be as quick as this:

[codesyntax lang=”bash” lines=”no”]

svn co http://svn.apache.org/repos/asf/incubator/wookie/trunk
ant run

[/codesyntax]

an example of a Running Wookie Instance
an example of a Running Wookie Instance in MSDOS cmd shell

Several lines of text later after SVN has copied you the latest trunk build of Wookie, and ant has performed a great many tasks including downloading IVY, Wookie will instantiate and start

logging stuff to the screen.

 

There are some pre-requisites here though:

  • An accessible JRE (Java Runtime Environment) on your machine
  • a copy of Apache ANT (for windows – you can check out WinANT which takes out some of the grief in getting this going and setting environment paths and so on, and has the added bonus that you can get an archived copy of WinANT 4.1 – the last 1.7.x release before ANT migrated to the 1.8 series that leaves your Wookie broken and downhearted).
  • Subversion
Wookie first menu
Wookie's first menu

Once it’s up and running you will have a nice fast localised copy of Wookie running with all the default settings (so the Admin user and password are java & java respectively).

You can fiddle with some settings in the config files that reside in the ‘root’ level of the trunk directory. Best place to get information on these is the wookie site.

 

Flushed with this success, I determined to remove the schizoid and out of date copy of Wookie we have running on the LRT server to see if a standalone copy might work better. The LRT server – like others at the MMU – is behind the MMU firewall. At the time I built it we were not allowed to be added to the bypass group that permits servers to communicate with the WWW without having to go via a proxy – which usually makes a mess of all sorts of things on a server level. So – the original install of Wookie had entries in the config to add the proxy in by hand – and – alas – down the line, it was just possibly getting in the way of examining accurate widget behaviour.

Nowadays the whole server is in the bypass group, so in theory a fresh install and a standalone server should have widgets that can communicate with resources out on the web without judicious proxy messing.

Wookie's Demo Widgets
Wookie's Demo Widgets

Et Voila – it works. We now have a ‘standalone copy’ running on a server hosting widgets that now actually tell me what the weather is in Manchester (and no, it’s actually sunny!).

Running with Wookie – Tips for maintaining a standalone install

The problem is, that we are running this is on an actual server. It’s designed to sit there without someone nurse-maiding an instance of a program running in a terminal or shell, any reboot, any premature death of that shell (like killing the SSH shell from my machine to the server in which I actually launched Wookie) is going to see Wookie die a death. On a reboot, it’s not going to come back alive.

The standalone doesn’t act like a standard daemon in Linux. Once you hit ant run – its going to run and stay resident in the shell window you typed that command in – logs to that window and no escape from it without killing wookie with a control-c.

So – how do we manage?

SCREEN is a wonderful tool that has, after some years, now wormed its way onto standard distros for *NIX and even MacOS X. It allows you to run virtual terminals that stay resident in memory, have full TTY access and a witty set of escape sequences that, at a base level, allow you to essentially kick a terminal into life, run a command and virtually exit the terminal. Next time you log in you can reuse the Screen command and attach to that very same terminal you left – and, barring a reboot of the box – it will still be there with your program running merrily away and not the least bit bothered that you wandered off and left it.

Check this command out:

[codesyntax lang=”bash” lines=”no” blockstate=”expanded”]

screen -S "wookie-running" -d -m ant run -Drun.args="initDB=false"

[/codesyntax]

Woah neddy! thats a lot of arguments, here is an explanation:

  • screen – the command we need
  • -S “wookie-running” our instance of screen will have a nice english language name to allow us to refer to it by
  • -d -m as a couplet – these commands allow screen to do its stuff, run your command, and automatically come back out to the terminal you are currently in – auto-detaching itself in other words. The cunning part here is that your stuff is still running in the background in memory
  • all the rest is the standard wookie command to run without destroying our existing DB

You can nip back into your virtual terminal by the following command:

[codesyntax lang=”bash” lines=”no” blockstate=”expanded”]

screen -r wookie-running

[/codesyntax]

except you need to swap wookie-running out to whatever name you used for the screen sessing with the -S command. Hey presto – you will shoot right into a log of log lines Wookie has been dutifully dumping to the Standard IO of the terminal.

Getting back out? A doddle, hold control A and hit c

This will take you back to your original terminal without killing the session. If you want to kill the session, hit control D, or control C to kill Wookie then then Control D to get out of the virtual screen.

You might take to this method of doing things, I’ll stick some links in to good resources relating to Screen as soon as possible, it is extremely powerful and indeed programmable too. I will leave you with one final very useful command in case you do decide to do a few screens:

How to see how many screens you have established:

[codesyntax lang=”bash” lines=”no” blockstate=”expanded”]

lrt ~ # screen -ls

There are screens on:

20486.wookie-running    (03/21/2011 10:58:06 PM)        (Detached)

17838.stats     (03/01/2011 10:25:58 AM)        (Detached)

15863.backup    (02/24/2011 09:30:26 AM)        (Detached)

3 Sockets in /var/run/screen/S-root.

[/codesyntax]

Finally, we haven’t really solved the issue of making sure Wookie is running following a server reboot. This isn’t a massive problem, so for the moment I will just suggest that the screen command can be put into a standard init shell script. It must either be properly pathed, or execute a ‘cd’ command to the Wookie trunk before executing the screen portion. I will trial this shortly and put the additional steps into this article.

Lessons Learned

Really, if you are testing something, it’s not always best to go for the platform you think you will need to run it in anger. Luckily for us we had time to adapt, chuck out the stuff we didn’t need and came back to the problem from a fresh angle.

If running the server itself is your main angle (maybe you are a sysadmin), then invest some time on the Wookie pages and the email group finding out the best combination of dependencies – JDK, ANT, JDBC etc… really consider if you even need MySQL vs the built-in Derby engine. These choices may change depending on the extent of your target audience and the numbers of hits you expect Wookie to be able to cope with. If you expect to hang a major system off the back of this – remember – it’s in Beta/ Incubation, hiccups are going to happen from time to time despite best efforts of the developers.

If, on the other hand, you are really more interested in developing the widgets themselves, just go with the easiest option and use the Wookie standalone. It’s far less messy, and you can be up and running developing widgets within ten minutes or so.

Caveats and other Witty Considerations

I have used the term standalone here to indicate the difference in how the platform behaves, rather than the notion that you could use Wookie in this way off the net. When you type ant run, ANT is going to try to nip off and find updates to IVY, and it will fail miserably if you aren’t on the net, or are sitting behind a proxy and haven’t set the correct proxy variable. Here’s a couple of pointers:

Running ANT in offline mode

[codesyntax lang=”bash” lines=”no”]

ant run -Doffline="true"

[/codesyntax]

 

Setting the Proxy variable for ANT (behind a proxy)

[codesyntax lang=”php” lines=”no”]

ant -run -autoproxy

[/codesyntax]

Read a whole lot more about Ant and proxy here

Due Thanks to…

Scott, Ross, Paul and Co

Please dip in with comments, this post was put together over a few days including two great days at the Bolton CETIS Widget Bash, so most likely there are some lapses of concentration! Corrections gratefully received.

Now, give it a go, give the guys a go and get active. Get hold of Wookie, read about Widgets, grab some, play with them, join the mailing list [info further down the page].

http://ant.apache.org/manual/proxy.html

Widget development

Over the last few months, W2C has been laying foundations for its extensible VLE by developing web services to publish frequently-asked-for information from university systems. We are now ready to test the potential of W3C Widgets as a mechanism for publishing the output of these web-services on a range of platforms, and have just written our first widget. This post describes our experience.

To de-scope security issues from our first development, we decided to develop a widget to publish public data available from our PC Availability web-service. First we reviewed published examples of widgets and found particularly helpful WIDE’s step-by-step guide to developing a Calendar web-service and Scott Wilson’s recent post on progressive enhancement of mobile content. Based on this initial review, we decided that our widget should use:

  • jQuery – javascript library (to consume our web-service)
  • jQuery-mobile – javascript library and CSS (to render the content)
  • A simple Controller model to organise our javascript functions

and we knew we’d need:

  • a Wookie server – Steve’s working on a post about lessons learned from getting this going!
  • a folder structure in which to develop our PC Availability widget, comprising
    • config.xml
    • pc.html
    • scripts folder, containing:
      • latest copy of jQuery.js (minimized version)
      • latest copy of jQuery-mobile.js (minimized version)
      • our javascript: pc.js
    • style folder, containing:
      • latest copy of jQuery-mobile.css (minimized version)
      • folder of jQuery-mobile images

Our files contained the following:

[codesyntax lang=”xml” title=”config.xml”]

<widget xmlns="http://www.w3.org/ns/widgets"
   id="http://mmu.ac.uk/pc"
   version="1.0"
   height="480"
   width="320"
   >
  <name>PC Availability</name>
  <description>A sample widget to display PC availability</description>
  <content src="pc.html"/>
  <author>Mark Stubbs</author>
</widget>

[/codesyntax]
Our config.xml file defines a simple “PC Availability” widget that will be loaded from pc.html. Default size has been set as 320×480 for mobile rendering.

[codesyntax lang=”xml” title=”pc.html”]

<html lang="en">
  <head>
	<meta charset="utf-8" />
	<title>PC Availability</title>
	<link rel="stylesheet" href="scripts/jquery.mobile-1.0a3.min.css" type="text/css" />
	<script type="text/javascript" src="scripts/jquery-1.5.1.min.js"></script>
	<script type="text/javascript" src="scripts/jquery.mobile-1.0a3.min.js"></script>
	<script type="text/javascript" src="scripts/pc.js"></script>
  </head>
  <body onLoad="Controller.init()">
    <div data-role="page" id="home">
      <div data-role="header">
        <h4>PC Availability</h4>
      </div>
      <div data-role="content" class="ui-content">
        <ul data-role="listview" id="rooms-listview" data-theme="d" data-inset="true">
        </ul>
      </div><!-- /content -->
    </div><!-- /page -->
  </body>
</html>

[/codesyntax]
Our pc.html file loads the jQuery-mobile 1.03a stylesheet; the jQuery 1.5.1 minimized library; the jQuery-mobile 1.0a3 minimized library and our own pc.js javascript. The body tag contains an onLoad handler that calls a javascript function we have defined for the Controller class called init(). The xhtml within the page is organized to present information as a single page (using the div data-role=”page” tag), which has a header (div data-role=”header”) and content (div data-role=”content”) section. Within the header section, the h4 tag is used for the page title. Within the content section, an unordered list tag is included (with an id of “rooms-listview”), which will contain list items for current PC Availability at each drop-in space within the university. The ul tag is styled with some jQuery-mobile markup: data-role=”listview” data-theme=”d” and data-inset=”true”. Further information about controlling the presentation of list data in jQuery-mobile is available at http://jquerymobile.com/test/#docs/content/../api/../lists/lists-themes.html.

 

[codesyntax lang=”javascript” title=”pc.js”]

var Controller = {
	init:function() {
		Controller.update();
	},

	update:function() {

		var loc = Widget.proxify("http://testapis.ad.mmu.ac.uk/icts/Service1.svc/getPcAvailability");

		$.ajax({
		        type: "GET",
			url: loc,
			dataType: "xml",
			timeout: 1000,
			complete: Controller.parseResponse
		});
	},

	parseResponse:function(response) {

		var rooms = $("#rooms-listview");
		rooms.empty();
		$(response.responseXML).find("room").each(function () {
			rooms.append($("<li/>").text($(this).attr("location") + ": "
			+ $(this).attr("free") + "/"
			+ $(this).attr("seats") + " free"));
		});
		rooms.listview("refresh");
	}
}

[/codesyntax]

Our scripts/pc.js file defines the Controller class referenced in the body onLoad handler of pc.html. The Controller class has 3 functions:

  1. init()

    initializes by simply calling the update function

  2. update()

    uses Wookie’s Widget.proxify to get a URL within the same domain (using Wookie’s proxy to avoid cross-domain scripting restrictions) that can be used to call the PC Availability web-service, which it then does using the jQuery $.ajax function; this calls the Controller’s parseResponse function when it completes, passing across the httpResponse received from the web-service call.

  3. parseResponse(response)

    gets a handle for the rooms-listview element, clears its content and then iterates over the “room” elements in the xml returned from the web-service. A function is defined for handling each room element that is found. That function appends to the rooms-listview element a new list-item and sets the text of the list-item to be the location followed by the number of seats free, followed by the total. The parseResponse function completes by refreshing the rooms-listview with this new content.

On our Windows 7 development machine, we selected the pc.html, config.xml and the scripts and styles folders and right-clicked on the “send to Compressed (zipped) folder” option. We then renamed the resulting file “pc.wgt“. From the Wookie Adninistration menu we clicked the “Add new widget”, browsed for our pc.wgt file and clicked the “Publish” button to add it. Our PC Availability widget (minus an icon, as we hadn’t specified one in the config.xml file) was then available in the widget gallery for testing. We added the address of our PC Availability web service to the Wookie white list (so that the Widget.proxify command would work for the URL we’d specified) and then clicked on the demo button in the gallery, which produced this in Firefox:

Screen-shot of PC Availaibility widget running in Wookie using Firefox 3.6.15
Screen-shot of PC Availaibility widget running in Wookie using Firefox 3.6.15

We then tried in IE8 and the widget was rendered without any CSS and a javascript error. After much searching we found this post about IE support in jQuery, which suggests that work still needs to be done for IE 7/8 and Windows Phone 7, but that Scott Jehl had produced an “at your own risk” workaround that must be loaded after the jQuery library and before the jQuery-mobile library loads. We inserted the javascript inline between the two library calls, and the widget then rendered in IE8:

Screen-shot of PC Availability Widget rendered in Wookie using IE8.0.7600
Screen-shot of PC Availability Widget rendered in Wookie using IE8.0.7600

Apart from not having rounded corners, there was no obvious difference between the rendering of the two in different browsers (content differs as expected from a dynamic feed).

We then tried in a number of other browsers:

  • Safari on web and iPhone worked
  • the Android browser worked on a phone and a Galaxy Tab
  • the native browser failed to display anything on a Blackberry or a Windows Mobile 7 phone
  • Opera mobile worked on a Blackberry

Our initial foray into the world of widgets was pretty positive, although support for popular Windows browsers would need to move beyond an “at your own risk” hack for the interesting progressive enhancement approach to be viable for production deployment. We hope this post will encourage others to have a go and share their thoughts.