Breaking Down Barriers: Grand Finale

In a nutshell

The project aimed to complete the following two tasks:

Task 1: Open-up 8 courses in the Landmap Learning Zone through Creative Commons BY-NC-SA and redesign the Learning Zone area of the Landmap website, linking through to the ELOGeo repository at Jorum.

Task 2: Transfer the hosting of the ELOGeo repository to Jorum from University of Nottingham ensuring the sustainability of the ELOGeo Project outputs. Technical solutions will be sought in developing a specific community repository site within Jorum which will be transferable to other communities that may have a similar requirement in the future.

Project outputs:

  • (Task 1) The project has exceeded the delivery of providing 8 courses in the Learning Zone through Creative Commons BY-NC-SA by providing an additional course called ‘Image Processing for GRASS GIS’. GRASS is one of the most well know open source freely available GIS softwares in the geospatial community and will be a welcomed addition to the other 6 image processing courses now openly available.
  • (Task 1) The Learning Zone redesign is now complete incorporating links to new content for the Open Educational Resources; Spatial Science for Schools (open to all) and ELOGeo repository
  • (Task 2) The transfer of the ELOGeo repository to Mimas from the University of Nottingham has been achieved including applying the new theme to the repository (blue colour) from the old theme (orange colour). The next step is to add this new theme into the Jorum DSpace 1.8 architecture. The overall aim is to produce a DSpace sub-site within Jorum for the GeoKnowledge Community branded as ELOGeo. This part of the work has been very challenging due to Jorum never before providing community specific sites within its’ service, with a number of technical issues causing hold-ups. The ELOGeo community specific site within Jorum will go live not long after the new Jorum service (using DSpace 1.8) launches in mid November 2012. A sneak preview of the new ELOGeo interface can be seen in the short video post below.

Showcase of outputs so far…

Impact for the GeoKnowledge Community
The following project impacts are as follows:

  • The international geospatial community wanting to gain knowledge on how to use satellite, airborne and other spatial datasets in their research/work can now obtain these key skills through the new Open Educational Resources available through the Landmap Learning Zone.
  • Lecturers seeking e-learning content to support their lectures can now refer to the OER materials at Landmap.
  • UK industries/local councils seeking materials for Continuing Professional Development (CPD) for new members of staff or staff who have not used spatial data before can now make use of the ‘open’ materials.
  • European and worldwide academic institutions seeking high quality courses on image processing for leading commercial software’s such as ENVI, ERDAS, Idrisi, Definiens and PCI now have access to the open ‘Image Processing Courses’, such courses can cost at least £1,000 per person to attend.
  • Promoting the importance of OER to the geospatial community at key events such as INSPIRE 2012, RSPSoc 2012 and the Geoforum 2012. Explaining how providing OER is a two way process, encouraging members of the audience to contribute content to the ELGeo repository and Learning Zone OER.
  • Providing the technical challenges of creating a sub site within DSpace 1.8 at Mimas for combining the ELOGeo repository with the Jorum repository through the project blog.

Project Recommendations

  • As JISC have also funded a project at the University of Nottingham to build on the work of the ELOGeo project at that location we would recommend additional funding is provided to set-up a mirroring infrastructure to keep automatically in sync the two ELOGeo repository instances (Jorum & Nottingham).
  • Usability testing of the ELOGeo interface within Jorum to obtain user feedback and establish areas of improvement.
  • Run a project to encourage further contributions to ELOGeo from the user community so that the facility is used regularly in the geospatial open data, open software and open standards arena.
  • Establish a  quality assurance system for ensuring materials are up-to-date and still relevant to the GeoKnowledge Community.

Editing a theme for ELOGeo

Adapting the Mirage Theme

The ELOGeo Project theme was based on a theme called ‘Mirage’ which features a left navigation and top breadcrumb trail. To this were added logos for the involved parties e.g. University of Nottingham and Mimas. The design used an orange and red colour scheme (Fig 1). The developers had utilised the Mirage theme which had not been renamed – just amended. To bring the ELOGeo repository into Jorum , we ran the risk that either the Mirage theme was already in use by another community or that because Mirage is a popular layout, another community may need to use the Mirage theme in the future. Our designers had moved away from the warm colours to a white, blue and grey scheme with green buttons (Fig 2). However, the actual layout is very similar to the older ELOGeo scheme with some alterations to spacing and other minor structural changes.

BeforeAndAfter

ELOGeo before and after

It would be necessary to adjust the main page layout adjust some style information and alter the structure of the HTML divs and in some cases assign ids or classes to the divs. Whilst doing this it made sense to rename our Mirage theme as ‘elogeo’ to reflect the repository name.

Renaming the theme

Renaming the theme, unfortunately was not as simple as just renaming the theme folder. Inside the theme directory is an XSL file named after the theme. This file name must match the directory name (and on some platforms this is case sensitive). It is also worth checking that the theme folder isn’t referenced anywhere in the theme files.

Some elements of the theme can be amended without rebuilding DSpace. Since the rebuild process can take a number of minutes, it is much faster to work on the live web application and copy back the changes to the source directory to be built later.

The XMLUI web application is based on the Apache Cocoon framework that incorporates caching. This cache must be cleared to view the development changes.

Edit the theme in [Tomcat Folder]/webapps/xmlui/themes/[themename]/lib/[xsl or CSS].

Clear the cache and restart Tomcat:

apache-tomcat-7.0.11/bin/shutdown.sh
rm -rf [Tomcat directory]/work/Catalina/[domain]/xmlui/cache-dir
apache-tomcat-7.0.11/bin/startup.sh

When DSpace is rebuilt again after any configuration changes, the webapps folder of the DSpace Build Directory will be overwritten. This means that the XSL and CSS style changes in the Tomcat webapps directory ([tomcat directory]/webapps will not have been incorporated in the new build. Changes to the live web application MUST be copied back from the Tomct webapps directory to the DSpace source directory.

cp -r [Tomcat Directory]/webapps/xmlui/themes/[theme name]  [DSpace source directory]/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/themes

General theme structure

A Manakin (XMLUI) theme is composed of XSL and CSS files which can be readily amended. More in-depth customisations require changes to the theme’s aspects – which in this case we haven’t had to do.

Editing the Main Layout

The main layout of the interface is defined in the file page-layout.xsl. This determines how the content is translated into HTML. Any additions or alterations such as the order of divs, changing or adding classes and ids and inserting any static text or images is done my editing this file. The working file is at [Tomcat Directory]/webapps/xmlui/themes/elogeo/lib/xsl/core and this must be copied back to the DSpace source at [DSpace Source directory]/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/themes/elogeo/lib/xsl/core.

The main CSS style information is in [Tomcat directory]/webapps/xmlui/themes/elogeo/lib/css/style.css and again this must be copied back to the DSpace source if the changes are to survive a rebuild:  [DSpace Source directory]/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/themes/elogeo/lib/css.

The Mirage theme main page structure accommodates a reasonable number of divs to contain the content. It can be difficult to work out which html structures are controlling elements such as the content width. In our case this was being set by the trail UL element width.

Viewing/Editing Styles

Opening the DSpace URL in Firefox gives the option under the menu Tools/Web Developer/Inspect to view the page structure and style. The style information can be edited in real-time. It is useful to see the structure of divs in the output and to identify which have an ID or class referenced by the CSS of the site.

Amongst other things, our design required moving the login link from the page header to the right hand side of the breadcrumb trail under the page header. Because the UL of the breadcrumb trail set the page width, these changes caused a number of issues with page flow. To assist in getting the page structure and flow correct, I built a simple test HTML page outside of DSpace with div ids mirroring the DSpace structure. I added inline style information under the style attributes of each element.

To accomplish the new design we had to insert some new divs into page-structure.xsl and to edit the style properties of a number of elements.When this page flow worked correctly, I updated the CSS attributes in the main style sheet.

Customising the DSpace front page

A new installation of DSpace and the XMLUI interface has a default front page containing information about DSpace. It is intended that this be replaced with a description of the actual repository. It is necessary to customise the front page news item and the default static text throughout the repository.

DSpace front page

The front page is configured as an XML file. The XML is similar to XHTML in some respects but the use of HTML tags is inconsistent. This data in this file is compiled into the DSpace web application during the build process. Most web browsers will attempt to format even bad XHTML, however DSpace does not use the file as XHTML but parses it as XML. This process requires correctly formatted XML. Should it be invalid XML such as missing closing <p> tags, the parser will silently fail and all the content will be ignored.

Although some XHTML markup works as expected, links as anchor (<a>) tags are not used. To include links, the following markup must be used:

<xref target=”http://myuni.edu/”>My University</xref>

The front page XML file is stored at:

[DSpace working directory]/config/news-xmlui.xml

Which needs to be copied back to:

[DSpace release directory]/dspace/config

Repository front page

Front page news item and message text configuration

DSpace default text

The DSpace application interface is built using a default list of text fragments stored in a resource file as constants. Our design incorporated the use of the repository name ELOGeo as opposed to the generic DSpace repository.

The XML configuration file is found in the following location.

[DSpace working folder]/webapps/xmlui/i18n/messages.xml

DSpace will need to be rebuilt for changes to take effect so the edited file must also be copied back to the source location at: [DSpace source directory]/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/i18n.

As with the Repository theme and asset files, we copied the messages.xml file from the source instance of DSpace.

One of the recurrent findings with this project is that we know what we want to change and we know how to change it – using images, CSS and XSL. The problem is finding where in DSpace we need to change files. Because DSpace sometimes needs to be rebuilt, we must also remember to copy back changes to the source code and the locations in the source are sometimes not the same as in the web application output.

Updating DSpace from another instance

Understanding where DSpace content is stored

DSpace repository content consists of a database and files known as assets. The database acts as a fast index and the asset object data is stored as files in their native format but without an extension.

In addition to importing the SQL dump into the database obtained from the University of Nottingham, we need to copy over the corresponding assets. Should there be a delay in obtaining the SQL dump and the assets it is possible that they may not be synchronised. It may be worth taking the source site down to prevent editing whilst exporting the content. The asset files contain item content but also other information such as licence agreements. Although we took the assets at the same time as the SQL dump, when cross checking, we found the number of asset folders was greater than the number of items in the SQL dump.

DSpace Installation Key Locations

There are three important locations in our DSpace installation:

  1. The source release directory.
  2. The DSpace build directory and
  3. The Tomcat directory.

1. DSpace Source directory

The location of the downloaded DSpace source code. After unpacking the full source download this folder will be named after the DSpace version number e.g. dspace-1.7.2-src-release/. Most changes to the DSpace configuration require that it is rebuilt.

Any changes to the working web application therefore must also be made in this location.

2. DSpace Build directory

The DSpace build directory is populated when DSpace is built or rebuilt from the source code. The DSpace Web application is output to the location specified in [DSpace source directory]/dspace/config/dspace.cfg. This can be thought of as being the compiled working application. The asset files with item content are stored here.

3. Tomcat directory

Named after the version of Apache Tomcat e.g. apache-tomcat-7.0.11. Tomcat is the Web Server that provides a Web interface to DSpace. In our implementation, the Web Application in the build folder is copied into the Tomcat webapps directory. (Copy the folder [DSpace build directory]/webapps/xmlui to [Tomcat directory]/webapps/). The XMLUI Web application interacts with the DSpace application in the build directory.

Tomcat can be configured to point directly at [DSpace Build Folder]/webapps. This would seem simpler than having the copy of webapps – but it does require changing the default Tomcat configuration.

Key Files

Asset Files

The ELOGeo asset files go under the [DSpace build directory]/assetstore/. Inside will be a number of numeric directories containing the repository content. When DSpace is rebuilt this content will not be deleted. It is not necessary nor a good idea to copy back the files to the [DSpace Source Directory] because the files in the Build Directory are the current live files.

Theme Files

Several themes are available for DSPace. The ELOGeo repository uses a variation on the Mirage theme. This determines the look and feel of the DSpace interface using XSL, JS and CSS files to control the layout and styling of the HTML markup. If any changes are made to the live Web application, it is important that they be copied back to the Source directory so they will be incorporated in future builds. The theme files will be present in thre locations:

  1.  Source Code: [DSpace source release directory]/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/themes
  2. Application: [DSpace build directory]/webapps/xmlui/themes
  3. Tomcat copy of application: [Tomcat directory]/webapps/xmlui/themes

To use a specific theme, in our case “elogeo”, the configuration file needs to be amended:

DSpace source directory]/dspace/config/xmlui.conf

This required theme needs to be the last entry in the <themes> element.

<theme name=”[Theme name]” regex=”.*” path=”[theme directory name]/” />

DSpace will need to be rebuilt for the changes to take effect. It is possible to edit the theme in use without rebuilding DSpace. This will be outlined in a later post on editing themes.

This work has enabled us to gain an understanding of how and where DSpace stored content and how the layout of its interface can be customised.

This demonstrates that there is a relatively easy way of copying one instance to another using the database dump, asset files and some configuration changes.

The relationship between the three file locations isn’t at first obvious. It is crucial to understand which files need to be copied back to the source code and which files shouldn’t be copied back to the source code so they can be incorporated into future builds.

Configuring Apache Tomcat and Building DSpace

Apache Tomcat – Web Access

Web access to DSpace is provided by Apache Tomcat via the XMLUI web application (also known as Manakin). This interface is one of those provided with DSpace. It is based on the Apache Cocoon architecture. To configure this we need to set up the Tomcat host and connector as follows:

The configuration file for Apache Tomcat is located in the Tomcat directory (patterned after the version number) in our case apache-tomcat-7.0.11.

In our case the domain is elo-geo.net but on a local installation this could be localhost.

[Tomcat Directory]/conf/server.xml

<Host name="[domain]"  appBase="webapps"…

We have pointed the server to the location of DSpace Web application (XMLUI) which we have copied to [Tomcat Directory]/webapps. This could equally point directly at our DSpace build directory like this:

<Host name="[domain]"  appBase="[dspace build directory]/webapps"…

Our hosting environment is shared within our organisation and this means that port numbers can be in use by other applications. The default port for Apache Tomcat is 8080. There are a number of Apache Tomcat instances running in this environment and port 8080 is often is use. The practical issue with this is that Tomcat will not always start up properly because the operating system cannot assign the specified port. We have changed this to 8089 for this installation of DSpace to avoid clashes.

The port number is defined in the server.xml configuration file as a connector. We amended the entry for the default 8080 port:

<Connector port="8089" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

If there is a suspicion that a port is in use, the traffic can be viewed on UNIX with the following command.

netstat -an | grep [port number]

Building and Rebuilding DSpace

The building of DSpace includes both Ant and Maven processes and finally copying the Webapps to the Tomcat directory and restarting Tomcat.

apache-tomcat-7.0.11/bin/shutdown.sh
#Build Maven Process
cd [DSpace source directory]/dspace
mvn package

#Build 2 Ant Process
cd target/dspace-1.7.2-build.dir/
ant -Dconfig=config/dspace.cfg update

#Back out to root directory
cd ../../../../
#Copy to Tomcat
cp -R [DSpace Build directory]/webapps/xmlui [Tomcat directory]/webapps

#Restart Tomcat:
[Tomcat-directory]/bin/shutdown.sh
[Tomcat-directory]/bin/startup.sh

#Check port usage
netstat -an | grep 8089

The build process will time stamp and retain some directories as a backup.

The rebuilding process is clear in the DSpace documentation. Assuming a clean installation and availability of the default Apache Tomcat port of 8080 this will work without problems. When a hardware environment is used that has a number of other users, Java applications and even instances of Tomcat there can be problems starting DSpace. Since the port number issue relates to the multiple users of Tomcat it is not really a DSpace issue, but nevertheless a common problem.

INSPIRE 2012

The INSPIRE 2012 Conference kicked off with a great two days of workshops. I especially enjoyed Debbie Wilson’s presentation ‘INSPIRE Essentials – Back to Basics’. This workshop had a few multiple choice questions to test the audiences knowledge on INSPIRE. One of the interesting points Debbie made was the fact that many geoportals have popped up on the scene but that there appears to be alot of redundancy and replication. This is one of the key problems INSPIRE is trying to address so that datasets are harmonised and knowledge can easily be shared between and within organisations.

INSPIRE 2012 Opening Plenary

Another workshop attended is M. Taner Aktas ‘Geospatial Knowledge Sharing and Transfer, Innovation & Future Trends in Geo-Information’. Taner explained  how the transfer of knowledge may not be the same as what is received but some modified version of it. This could be because of: lack of trust; different cultures, language, mental mode; lack of time and meeting places; status and rewards issues or lack of absorptive capacity.

Tomorrow I will be presenting a poster on ‘Breaking Down Barriers – Building a GeoKnowledge Community’ to help promote the work being achieved as part of this JISC funded Rapid Innovations Project.To see a PDF of the poster please view the link below.

INSPIRE-A1-Poster

Database migration

Following my last post, we investigated the import/export mechanism, and discovered that the database dump from Nottingham University will be easier to import into the Postgres database at the test version of DSpace 1.7.2.

After much discussion and effort we managed to get the database dump in sql format which is around 2 gb in size. Following commands were executed to load the data and then the dspace user was updated to elogeo in <user directory>/elogeodspace/config/dspace.cfg file. The Apache and tomcat were stopped and restarted.

-bash-3.2$ createuser -U zzelogeo -d -P
Enter name of role to add: elogeo
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y
-bash-3.2$ createuser -U zzelogeo -d -P postgres
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y
-bash-3.2$ createdb -E UTF8 -T template0  dspacelm

-bash-3.2$ psql dspacelm < <user directory>/db_dump/elogeodump.sql

We noticed there is a memory issue, so we ran the following command, stopped and restarted Tomcat, but it did not help much.

-bash-3.2$ JAVA_OPTS=-Xmx2048m

This is a very common Java problem of heap space and we should be able to sort this out very soon. We will keep you posted on further developments.