About dulcet_bg

Applications Developer, Mimas, The University of Manchester

Database migration

Following my last post, we investigated the import/export mechanism, and discovered that the database dump from Nottingham University will be easier to import into the Postgres database at the test version of DSpace 1.7.2.

After much discussion and effort we managed to get the database dump in sql format which is around 2 gb in size. Following commands were executed to load the data and then the dspace user was updated to elogeo in <user directory>/elogeodspace/config/dspace.cfg file. The Apache and tomcat were stopped and restarted.

-bash-3.2$ createuser -U zzelogeo -d -P
Enter name of role to add: elogeo
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y
-bash-3.2$ createuser -U zzelogeo -d -P postgres
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y
-bash-3.2$ createdb -E UTF8 -T template0  dspacelm

-bash-3.2$ psql dspacelm < <user directory>/db_dump/elogeodump.sql

We noticed there is a memory issue, so we ran the following command, stopped and restarted Tomcat, but it did not help much.

-bash-3.2$ JAVA_OPTS=-Xmx2048m

This is a very common Java problem of heap space and we should be able to sort this out very soon. We will keep you posted on further developments.

Advertisements

Data plans

Amir from Nottingham University has shared some documentation to import and export the items for the DSpace repository.

Basically the preliminary plan is to try testing a bulk import on Mimas front, and then get the items from Nottingham. There is a need to understand this export/import mechanism well which is not documented very well.

Once we get a database dump, we will try to setup a mechanism to be in sync with any data updates/uploads so that both Nottingham and Mimas DSpace sites are mirroring each other.

Watch this space for more about the data migration.

Earlier experiments

Hello this is me Bharti, one of the developers at Mimas. I have earlier been involved in setup of DSpace installation on which the current ELO-Geo repository is based at the University of Nottingham. We got a unix (Solaris 10) based server setup with following details:
Basic configuration:
4GB RAM, 80GB or more Disk space, 2 processors

Then the basic software that were installed were:
1- Apache HTTP Server 2
2- Java JDK 6 or later
3- Apache Tomcat 7.0.12
4- PostgreSQL 8.4
5- PHP 5.3.6
6- Apache Maven 2.2.x
7- Apache Ant 1.7 or later

Once the above got installed I needed to begin the process of installing the DSpace repository software and MediaWiki.
8- DSpace
9- MediaWiki

The Nottingham folks were not much familiar with the last two and not a server expert, only had experience of installing on a local Windows server. Thus it was a challenge installing java based DSpace software on Solaris, and much tweaking was required.

I got finally DSpace running by following the steps which I coudln’t find anywhere, so thought to blog:

========================================================================

-bash-3.2$ ./configure –prefix=<user directory>/postgresql-8.4.8 –without-readline
-bash-3.2$ gmake -C <user directory>/postgresql-8.4.8/doc all
-bash-3.2$ gmake -C <user directory>/postgresql-8.4.8/src all
-bash-3.2$ gmake -C <user directory>/postgresql-8.4.8/src/backend all
-bash-3.2$ gmake -C <user directory>/postgresql-8.4.8/src/backend/utils fmgroids.h
-bash-3.2$ gmake check
-bash-3.2$ gmake install
-bash-3.2$ initdb -D <user directory>/postgresql-8.4.8/data
————————————————————————————————————————–

Added the following to profile file under <user directory>

export ANT_HOME=<user directory>/apache-ant-1.8.2
export M2_HOME=<user directory>/apache-maven-2.2.1
PATH=<user directory>/jdk1.6.0_18/bin:<user directory>/apache-ant-1.8.2/bin:<user directory>/apache-maven-2.2.1/bin:<user directory>/postgresql-8.4.8/bin

-bash-3.2$ ./configure –prefix=<user directory>/postgresql-8.4.8/ –without-readline –with-pgport=5642

-bash-3.2$ gmake -C <user directory>/postgresql-8.4.8/
-bash-3.2$ gmake check
-bash-3.2$ gmake install
-bash-3.2$ <user directory>/postgresql-8.4.8/bin/initdb -D <user directory>/postgresql-8.4.8/data
-bash-3.2$ <user directory>/postgresql-8.4.8/bin/pg_ctl -D <user directory>/postgresql-8.4.8/data -l logfile start
-bash-3.2$ <user directory>/postgresql-8.4.8/bin/pg_ctl -D <user directory>/postgresql-8.4.8/data -l logfile stop

Change in <user directory>/data/postgres.conf

listen_addresses = 127.0.0.1

Stops for trying for IPv6

Add in <user directory>/.profile
export ANT_HOME=<user directory>/apache-ant-1.8.2
PATH=<user directory>/apache-ant-1.8.2/bin:<user directory>/postgresql-8.4.8/bin:

-bash-3.2$ createuser -U <user> -d -P <password>
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y

-bash-3.2$ createdb -E UTF8 -T template0  dspacelm
-bash-3.2$ psql -l

List of databases
Name    |  Owner   | Encoding  | Collation | Ctype |   Access privileges
———–+———-+———–+———–+——-+———————–
dspacelm  | <user> | UTF8      | C         | C     |
postgres  | <user> | SQL_ASCII | C         | C     |
template0 | <user> | SQL_ASCII | C         | C     | =c/<user>
: <user>=CTc/<user>
template1 | <user> | SQL_ASCII | C         | C     | =c/<user>
: <user>=CTc/<user>
(4 rows)

-bash-3.2$ cd <user directory>
-bash-3.2$ mkdir <dspace directory>
-bash-3.2$ chown <user> <dspace directory>
-bash-3.2$ cd <user directory>/dspace-1.7.2-src-release/dspace
-bash-3.2$ mvn –e package
Gave build failed error

Commented out the following code from the pom.xml the build completed successfully.

<!–
Builds DSpace Solr Stats for DSpace
–>
<!– <profile>
<id>dspace-stats</id>
<activation>
<file>
<exists>../dspace-stats/pom.xml</exists>
</file>
</activation>
<modules>
<module>../dspace-stats</module>
</modules>
</profile>
–>

-bash-3.2$ mvn –e package
-bash-3.2$ cd target/dspace-1.7.2-build.dir
-bash-3.2$ ant fresh_install

Got error – FileNotFoundException for database_schema.sql

Copied database_schema.sql from

<user directory>/dspace-1.7.2-src-release/dspace/etc/postgres

to

<user directory>/dspace-1.7.2-src-release/dspace/etc

-bash-3.2$ cd <user directory>/dspace-1.7.2-src-release/dspace

-bash-3.2$ mvn –e package
-bash-3.2$ ant fresh_install

Got error invalid db.name

Tried to change the installation directory to <user directory>/<dspace directory>
-bash-3.2$ cd <user directory>/dspace-1.7.2-src-release/dspace
-bash-3.2$ mvn –e package
-bash-3.2$ ant fresh_install

Got other errors

Change the db.name in dspace.conf file to postgres
Set the db.schema in dspace.conf file to

<user directory>/dspace-1.7.2-src-release/dspace/etc/posgres

-bash-3.2$ cd <user directory>/dspace-1.7.2-src-release/dspace
-bash-3.2$ mvn –e clean package
-bash-3.2$ ant fresh_install

Copied from

<user directory>/elogeodspace/webapps to <user directory>/apache-tomcat-7.0.19/webapps

-bash-3.2$ cd <user directory>/<dspace directory>/bin
-bash-3.2$ dspace create-administrator

Creating an initial administrator account
E-mail address: <email>
First name: <first name>
Last name: <last name>
WARNING: Password will appear on-screen.
Password: <password>
Again to confirm: <password>
Is the above data correct? (y or n): y
Administrator account created

Run apache
-bash-3.2$ cd <user directory>/apache/bin
-bash-3.2$ sudo ./apachectl start

Run tomcat
-bash-3.2$ cd <user directory>/apache-tomcat-7.0.19/bin
-bash-3.2$ startup.sh

=========================
Change the domain name

Change from <original> to <new> in following files:

1) <user directory>/apache-tomcat-7.0.11/conf/server.xml
2) <user directory>/apache/conf/httpd.conf
3) <user directory>/elogeodspace/config/dspace.cfg

========================================================================

The initial URL is http://elo-geo.net/xmlui

In case there is a 503 error, the following commands need to be followed:

-bash-3.2$ <user directory>/postgresql-8.4.8/bin/pg_ctl -D <user directory>/postgresql-8.4.8/data -l logfile stop

 

-bash-3.2$ cd <user directory>/apache/bin/

 

-bash-3.2$ sudo ./apachectl stop

This will ask for the user password

 

-bash-3.2$ sudo ./apachectl start

 

-bash-3.2$ cd <user directory>/apache-tomcat-7.0.11/bin

 

-bash-3.2$ startup.sh

 

-bash-3.2$ <user directory>/postgresql-8.4.8/bin/pg_ctl -D <user directory>/postgresql-8.4.8/data -l logfile start

Then the task was to customise DSpace and Mediawiki according to Nottingham copy of ELO-Geo.

We as part of this project of Breaking barriers, will now proceed to complete the process and enhance as per the project outlines.