Author: Joss Winn

I work in the School of Education at the University of Lincoln.

I’ve just upgraded jiscpress.org to WPM…

Joss Winn / / Microblog

I’ve just upgraded jiscpress.org to WPMU 2.8.1 beta. It’s plenty stable enough to develop from and uses a new widget framework, among other changes. It’s sync’d with WP 2.8.1 and you can read about changes in WP 2.8 here: http://codex.wordpress.org/Version_2.8 It’s likely that v2.8.x will be around for most if not all of the JISCPress project but we should keep an eye on v2.9 development to ensure that JISCPress remains compatible with changes in core. http://codex.wordpress.org/Version_2.9

Agile methodologies and open source development

Joss Winn / / Blog

In the course of writing the JISCPress Project Plan, I’ve been thinking again about our project methodology. The original funding call asked for projects to adopt an agile methodology like SCRUM or XP, which I am familiar with. We attempted to use XP while I was working at Amnesty International (not long after half the IT department were trained in Prince2!) and like any methodology, it was used in part rather than in whole. We  collected user stories, held five minute stand up meetings each day and released often and iteratively so that users could feed back on the product. ((It may look like an empty repository but over 20,000 assets are available to logged in Amnesty staff.))

The JISCPress project has four team members, including myself. None of us work on JISCPress full-time, having other work and study commitments. Equally, none of us work together in the same department and only Alex and I work in the same university. Alex is a student of computing at Lincoln, Tony lives on the Isle of Wight and Eddie lives in San Francisco. In addition to this, we’re working wholly with existing open source software (WordPress) that is openly developed and it has never been an option in my mind, to enjoy the benefits of that community but not attempt to contribute back using the same transparency of process. It was also proposed in our funding bid that “the project will seek to promote openness and collaboration from the point of bid announcements onwards.” By this, I was thinking in terms of the open source development process I have seen with WordPress and other projects where asynchronous discussion and contributions take place through mailing lists, IRC, a code repository, issue tracker and a wiki.

Reading the excellent OSS Watch website, I came across a page about the sustainability of projects and open development, and was particularly interested to read a quote from Gianugo Rabellino, CEO of SourceSense:

“If you think that one of the key ideas of agile is the unity of time and location – you need to be in the same place at the same time and doing a lot of discussion face-to-face – and then you have open development which is based on asynchronous, distributed working etc., then it looks like oil and water – they don’t mix”.

This is what I’ve been thinking recently, too. It’s not that they are wholly incompatible methods of developing software, but from what I know about agile methods, there is an assumption that the developers are working together in the same physical location, focused intensively on the same client driven product.

“Scrum enables the creation of self-organizing teams by encouraging colocation of all team members, and verbal communication across all team members and disciplines that are involved in the project.” ((Wikipedia: SCRUM))

Frankly, this way of working is impossible for us. On the other hand, projects that are openly developed often don’t have clients but instead have ‘communities’ of users. They rarely have short code sprints, they have open version-controlled repositories that allow anyone to test the code at any time. It’s worth noting that WordPress recently held a code sprint but given the size of the community, there were relatively few contributions. Many contributors work asynchronously and have other commitments over the course of their day, volunteering their time and effort when they can.

Likewise, JISCPress is intended to serve a community rather than a single client. We hope that it is the JISC community who lead the direction of the project through testing and feedback and who eventually benefit the most from the project. Beyond the JISC community, there is the wider community of users of WordPress and CommentPress who will likewise benefit from the project.

Ross Gardler, OSS Watch manager, describes the Open Development Methodology (ODM) as “a way for distributed team members to collaboratively develop a shared resource in a managed and sustainable way.” The ODM is characterised by:

  1. User engagement
  2. Transparency
  3. Collaboration
  4. Agility
  5. Sustainability

Agility and user engagement are also found in SCRUM and XP, but there is no requirement in these methodologies to be transparent, sustainable beyond the client’s specific use for the product or cater for a diverse group of asynchronous contributors.

With this in mind, I will continue to learn about and pursue an open development methodology for JISCPress because it is appropriate for our project. It is already part of an existing (WordPress) open development community and we have, from the start of WriteToReply and then the #jiscri call, placed a great deal of emphasis on openness and transparency of process.

It is too early in the project to measure the effectiveness of this approach. Eddie and Alex only joined us in the last few days and we’re still setting up the basic platform for working with. I have noticed that the use of IRC has not taken off despite my fondness for it. This is partly because all of us use GMail and tend to use Google Chat for quick conversations when we see we’re online at the same time, rather than having an IRC client open. Tony and I have an established way of communicating with each other over Twitter, which is public but a poor method of establishing context for the project as Twitter doesn’t archive tweets long-term and searching for anything seems to be hit and miss. I would like to establish weekly IRC meetings soon though. There is also the issue of working in a significantly different timezone to Eddie. IRC is for synchronous chat and when Eddie is at work,the rest of us are thinking of sleep. Eddie is talking about visiting the UK for a few days (paid for out of his own pocket), and I hope that the four of us and anyone else that is interested, will meet up for a day’s discussion and development.

There are clearly still things to be worked out and a routine to establish that works best for us, but I am keen that if a methodology is to be identified for the project, it is one of ‘open development’ rather than ‘agile’. I intend to devote a lot of my time on the project to ensuring that the wider WordPress community are aware of what we are doing and that they are welcome to contribute in any way they can. I shall write more about how we are addressing Ross’ five characteristics of an Open Development Methodology and am keen hear from anyone who has an opinion on any of this, including members of the JISCPress team, who I haven’t consulted before writing any of this.

I’ve just chatted over IM with Eddie. H…

Joss Winn / / Microblog

I’ve just chatted over IM with Eddie. He’ll be starting tomorrow. Here’s his outline task list for the next month: http://bit.ly/5Yp0K Basically, we hope to have a ‘bug-free’ (you know what I mean!) WCAG compliant version of JISCPress with author RSS feeds delivered by the end of the month. Paragraph embedding and paragraph comments by trackbacks might also get a look in.

Project SWOTing

Joss Winn / / Blog

One of the JISCRI project reporting requirements is a SWOT analysis of each project. It makes sense to attempt our first SWOT analysis sooner rather than later and update it using the comment form as we work through the project. Your comments are very welcome. Surely a bad SWOT analysis is one undertaken by a single individual (like this one!)

SWOT diagram

I think one of the main strengths of the JISCPress project is that we’ve effectively been developing it since February, when Tony and I set up WriteToReply. JISCPress is basically a re-thinking, re-working and further development of WriteToReply for a specific community and we can apply the lessons learned through WriteToReply, to the planning and development of JISCPress. In this sense, we’ve got a decent head start really and both Tony and I know where our own strengths and weaknesses, as well as our own particular interests lie in the project.

Another strength worth highlighting is the range of skills we bring to the project. I’ve been running different WordPress MU installations here at the University of Lincoln for the last year and have several years’ experience tinkering with Linux servers. I’m finding working on AWS to be an enjoyable and welcome learning experience.  Tony prefers to stay away from the bash command line, instead focusing on the way the data published on JISCPress can be repurposed, cross-referenced, syndicated and mashed up with other web services. He’ll also be looking at what value can be gained from the Google Analytics and Piwik APIs.

Anyone that knows a bit about CommentPress (now called ‘Marginalia‘), will understand that Eddie brings an excellent understanding of WordPress code to the project as well as some pretty advanced javascript skills. Eddie has always led the development of CommentPress/Marginalia and as it provides core JISCPress functionality such as paragraph level URIs and commenting, it’s a strength of the project that we have him on board. Were he not on board and we were reliant on another developer to work on CommentPress code, I would consider this a risk to the project. To quote from his site:

I believe in rapid prototyping, open-source, collaborations instead of competition, quick releases, smalls teams, debate, creative thinking, and transparency.

That’s exactly the type of person we need to work on this project.

Finally, Alex is a keen student of computing at the University of Lincoln with good PHP skills. As a student, he’s flexible with his time and not wholly reliant on the project for his income and it’s reassuring to have him working locally (and in the same time zone!). All in all, I think we’ve got a good spread of skills and interests on the team.

While I’m thinking of strengths, I’m confident that using Amazon’s infrastructure to work on the project will prove to be a strength. It allows us to work on the project in an environment that is independent of the University of Lincoln’s IT infrastructure.  I’m very lucky here at Lincoln to have root access to my own Linux server to work on, despite not being a member of the ICT department. I’ve no complaint at all about our ICT department and enjoy working with them, but on a rapid project like JISCPress with four team members working independently and the potential for contributions from the open source community, I’m pleased that we have our own space to work and I don’t need to bother my IT colleagues to restart the virtual machine or make changes to DNS records.

On the other hand, the membership of the team could be seen as a weakness. We’re not a tight team of developers working in the same institution but rather relative strangers working, for the most part, remotely and in Eddie’s case in a considerably different time zone. This could result in poor communication and lack of motivation if we let it and I hope that the pillars of communication in open source projects that we’ve set up (IRC, mailing list, code repository, wiki, blog) will help us stay in touch and motivated.  However, one of the main benefits of this project for both me and my employer, is being able to test this way of working on development projects. We don’t have an in-house team of web developers who could be pulled into this project and as much as I’d like my department to hire a researcher/developer or two, it’s not going to happen. So in order to work on JISCRI and similar funded projects, I need to show that this is an effective way of working. I hope it succeeds, because I like working on these types of projects and in order to innovate in our use of technology to support research, teaching and learning, we need to have the experience and capacity to undertake proper R&D and not just theorise about the potential of technology in the HE sector.

A threat to the project is that JISCPress is principally a tool for JISC document authors to publish funding calls and JISC project managers to publish their final reports. We need their buy-in to the project, not only to make it feel worthwhile but also to steer the direction of feature development. JISCPress might be seen as complicating JISC employees’ work, pushing something on them that they never asked for. It might also be seen as yet another requirement from JISC to Project Managers. I take this threat seriously, but I don’t let it worry me too much. JISC has made the decision to fund JISCPress as a ‘demonstrator prototype’ and there’s no obligation for them to put it into production use. They also recognise that we’re building a platform that could equally be of value to other organisations. WriteToReply and JISCPress are just two examples of what we’re developing. WordPress is a popular CMS and the work on Marginalia and additional features that we’ll be developing, can be cherry-picked or taken wholesale and put to good use. All code is developed under a GPL or compatible license. (Note that this has to be the case, because we’re developing for WordPress which is licensed under the GPL, calling functions in WordPress core code – not all WP plugin and theme developers understand this!)

Finally, for now, the project provides opportunities for anyone to get involved and in turn, by working in public on an open source project, I hope we’ll attract others who like what they see and want to contribute in any way at all. Comment, test, review and contribute code, if you can. Join the mailing list and introduce yourself. Working this way, with an emphasis on openness and transparency, I hope that opportunities arise that we don’t yet know about. One that we do know about is Google Wave, due out in September, and if we keep that in the back of our mind, there might be an opportunity to exploit this new and exciting platform and protocol. Maybe we’ll develop a JISCPress gadget for Wave that allows realtime comment and discussion on a document from Wave? Maybe JISCPress will largely become a ‘hidden’ CMS that is used exclusively by via publish and subscribe protocols such as RSS, AtomPub, XML-RPC, and Wave/XMPP?

Setting up JISCPress on Amazon Web Services

Joss Winn / / Blog

I’ve spent the last couple of days – about 13 hrs altogether – setting up JISCPress on Amazon Web Services (AWS). Prior to yesterday, I’d not really used AWS except for setting up the command line tools and starting and stopping a server, also known as an Amazon Machine Instance (AMI).

It’s gone pretty well and I’ve documented the outline of the process I went through on the JISCPress wiki. I used a combination of the Amazon Management Console and the command line tools to work on the Elastic Compute Cloud (EC2). To create ‘buckets’ on the Simple Storage Service (S3), I used the S3 Firefox plugin. There are a lot of third-party tools to interface with both EC2 and S3.

I work on virtual servers both at the university and on Slicehost, where WriteToReply is hosted. A server on AWS is a different kind of virtual server, which takes a little while to understand, but the documentation is good and I pretty much followed the suggested workflow.

In addition to using EC2 and S3, I am also using an Elastic IP address and the Elastic Block Store (EBS). The elastic IP address is convenient in that it allows you to ‘own’ a static IP address that you can bind your DNS A record to – in our case jiscpress.org Without the Elastic IP address, the IP address of the AMI is lost when you terminate the machine (turn it off) and so you have to change the DNS record and wait for DNS to re-propagate. It’s ‘elastic’ because while it is a persistent, external IP address, you can hot swap the machines that use the IP address. That was my first lesson.

My second lesson was to understand that there are a number of alternative, trusted public AMIs available to choose from. I don’t really mind which flavour of Linux I use and when looking at the AMIs that Amazon provide, I saw that only Fedora 8 was available so I chose that.  As it happens, there are also Ubuntu AMIs from Canonical.  Instead, I just upgraded Fedora 8 to Fedora 11 which wasn’t too much trouble, but had I known, I’d have just chosen an Ubuntu image from Canonical as they are more up-to-date. I also learned that despite upgrading the machine, the kernel remains the same. Amazon build kernels for their service and you can build kernel modules from the sources which they provide if you need to.

My third lesson was that although you can reboot an AMI just as you can reboot any virtual server, if you ‘terminate’ or turn off the AMI, you appear to lose all data that has been created since you created the AMI. In my case, that wasn’t much as I wondered about the persistence of data and had created my AMI after I’d got a basic web server with WordPress MU set up. But it’s really worth noting this as not only might you want to turn the machine off to save on running costs, but there’s also the chance that it might unexpectedly go down and you’d have lost your work. This underlines how AWS is being used. Machines are cheap and replicable. Use S3 and EBS for data you care about. Using a single machine for both production and development is never the right way to go about working long-term, but with a decent back-up strategy, it should work fine for us.

This led me sort out backups and I set up rsync to backup /var /home /root and /etc to an Elastic Block Store. EBS is a virtual block device (i.e. hard disk) which you can format and then attach and mount on your AMI. So I’ve got rsync backing up to /mnt/data

Getting the domain name and DNS sorted out was very simple. I registered jiscpress.org via Dreamhost ($10/yr) and then used a free UK-based DNS host to host the record. WordPress MU can run using either sub-directories (i.e. http://jiscpress.org/site1 or on sub-domains (i.e. http://site1.jiscpress.org). On the whole, it’s better to set up wildcard DNS and go with sub-domains, which I did by simply adding an A record entry of *.jiscpress.org against the Elastic IP address and ‘ServerAlias *.jiscpress.org’ in the section of the apache config file.

Finally, it looks like sending mail from an AMI is not as simple as you might expect. This is because the hostname for your machine is provided dynamically when you activate it and can’t be changed. This means that you can’t add a PTR record in your DNS and therefore can’t set up reverse DNS.  Without this, most mail hosts such as Hotmail or Yahoo, will treat mail from your server as spam. So far, Google is treating mail from the server as ‘neutral’ and letting it through. The simplest way around this is to relay your mail to an external relayer which a lot of AWS users appear to be doing. For the time-being, I’m not too worried about this but I may have to do more work on it if we find that mail is regularly failing to get through.

I’ve enjoyed the learning process of setting JISCPress up on AWS. I’ve only really scratched the surface of what the platform offers but once I’d got my head around how the different services work together, it seems pretty straight forward. The basic machine (1.2  GHz 2007 Opteron or 2007 Xeon processor, 1.7 GB RAM, 160GB storage) feels very, very fast, as does the network it’s running on. WPMU can be pretty resource hungry, but for the purposes of our project, I think this will be sufficient.

Anyhow, http://jiscpress.org is now live and running a bare WPMU install. I’ll refine it over the next week in preparation for Eddie and Alex to begin work in early July.

If you’ve got experience working on AWS and can clarify or correct any of my assumptions, please do. I get the feeling that now JISCPress is in the cloud, I need to relax a bit and enjoy the flexibility of the platform and learn more about what it has to offer.

JISCPress: A document discussion platform

Joss Winn / / Blog

We’re very pleased to announce that JISC have agreed to fund JISCPress, a six-month, £32,500 project led by the University of Lincoln, in partnership with the Open University and based on WriteToReply. JISCPress will provide a scalable community platform for publishing and discussing project calls and final reports, in order to support the grant bidding and project dissemination processes.

As you may know, WriteToReply is run in our spare time – lots of late nights and busy lunchtimes. Since launching the re-publication of the Digital Britain – Interim Report, we’ve been looking for ways to bring benefits from our work on WriteToReply, into the Higher Education community where we work. JISC fund much of the UK development and innovation in the use of ICT in teaching and research and in March, announced their Rapid Innovations funding call.

We quickly re-published the call on WriteToReply to demonstrate the benefits of publishing funding calls in this way and then went on to submit a bid which proposed a community platform for the JISC funding call process, based on our experience of setting up and running WriteToReply. As with WriteToReply, this will be an open, public project and all documentation and code will be available under open licenses.

JISCPress is a platform aimed at people working in UK Higher Education, but the platform itself could be easily adapted for other uses, just as WriteToReply is primarily focused on government consultation documents. The final platform will be available as an Amazon Machine Image so anyone will be able to host their own multi-document discussion platform with all the benefits you see on WriteToReply plus the additional features we’ll be developing throughout this project. We’re already advocating the use the platform in our own universities for the open (and closed) discussion of institutional strategies, for the critique of texts by students and for peer-review of research papers. What might you use it for?

Over on the JISCPress project blog, you’ll find links to a mailing list, wiki and code repository. Feel free to join us if this WriteToReply spin-off appeals to you. If you know anyone that might be interested, please do let them know.

You’re probably already aware that WriteToReply uses WordPress Multi-User and CommentPress. Eddie Tejeda, the developer of CommentPress will be working with us on the project and this will result in significant further development of CommentPress 2. So, if you’re interested in WPMU and CommentPress (as many people are), please consider following, contributing to and testing JISCPress.

We should also note that while the project is a spin-off of our work on WriteToReply, neither Tony or Joss are personally receiving any funds from JISC.  The contributions from JISC to cover our time on this project are paid directly to our employers and does not result in any financial benefit to us or WriteToReply (which is in the process of being formalised as a non-profit business).  In other words, while WriteToReply is a personal project, JISCPress is part of our normal work as employees of our universities (both Tony and I are expected to routinely bid and win project funds – you get used to it after a while!). Money has been allocated to fund dedicated developer time to the project, which will pay Eddie and Alex, a student at the University of Lincoln, for their work as freelancers.

Anyway, on with the project! Here’s the outline from our original bid document:

This project will deliver a demonstrator prototype publishing platform for the JISC funding call and dissemination process. It will seek to show how WordPress Multi-User (WPMU) can be used as an effective document authoring, publishing, discussion and syndication platform for JISC’s funding calls and final project reports, and demonstrate how the cumulative effect of publishing this way will lead to an improved platform for the discovery and dissemination of grant-related information and project outputs. In so doing, we hope to provide a means by which JISC project investigators can more effectively discover, and hence build on, related JISC projects. In general, the project will seek to promote openness and collaboration from the point of bid announcements onwards.

The proposed platform is inspired and informed by WriteToReply, a service developed by the principle project staff (Joss Winn and Tony Hirst) in Spring 2009 which re-publishes consultation documents for public comment and allows anyone to re-publish a document for comment by their target community. In our view, this model of publishing meets many of the intended benefits and deliverables of the Rapid Innovation call and Information Environment Programme. The project will exploit well understood and popular open source technologies to implement an alternative infrastructure that enables new processes of funding-related content creation, improves communication around funding calls and enables web-centric methods of dissemination and content re-use. The platform will be extensible and could therefore be the object of further future development by the HE developer community through the creation of plugins that provide desired functionality in the future.

Subject to user requirements, our planned project deliverables are:

  • A WordPress Multi-User based platform for authoring and publishing JISC funding calls in a form that allows paragraph-level comment and discussion either locally or remotely.
  • A meta-site that aggregates all document data into a single site for search, navigation by categories and tags and can syndicate searches, tags and categories.
  • Develop CommentPress to meet WCAG 2.0 accessibility guidelines, meeting public sector requirements.
  • Evaluation and integration of “related content” utilities to dynamically link related project calls and reports based on content and/or semantic analysis.
  • Evaluation and possible integration of remote, realtime messaging services such as Twitter and XMPP integration.
  • Evaluation and possible integration of enterprise authentication services such as LDAP and Shibboleth.
  • Evaluation and possible integration of OpenCalais, a semantic tagging service.
  • Documentation on how to exploit the benefits of AWS and clone the project instance for other uses.
  • A documented suggested workflow for document authors
  • Documented examples of how to fully exploit the platform for data extraction and syndication.
  • Documented ‘user stories’ for the JISC funding call process.

If this sounds interesting, please do take a look at the full project proposal and join us on the mailing list.