M718Report

= M7.18 - First integration phase complete =


 * Due: 29 November 2013
 * Delivered: 27 November 2013
 * Purpose: To integrate the appropriate workpackage developed services with Scratchpads.
 * Benefit: To enable seamless access to the services by users, thereby encouraging take up of the services.

Introduction
This milestone reports implementation of sustainable links between the tools developed by this work package (see bibliography service and mark-up services and Scratchpads. Hence the links are more than just proof-of-concept prototypes, they are robust, proven links that can be deployed across Scratchpads.

D7.1 - Community contributed bibliography
Deliverable report

This product is realised as RefBank.

From Scratchpads to RefBank

We have implemented a Scratchpads harvester that automatically harvests bibliographic entries from Scratchpads, and loads them into Scratchpads. The harvester is scheduled to run regularly and is hosted by Plazi. The harvester includes routines to prevent repeated work, detecting if no new entires have been added to a Scratchpad since it was last run, for example. Duplicate entries are not added to RefBank as they are detected as such on load and the new entry rejected.

Therefore, any researcher who loads their reference collection into a Scratchpad will automatically contribute their references to the wider biodiversity research community with no further effort required on their part.

The code is available from https://git.scratchpads.eu/v/scratchpads-harvester.git.

From RefBank to Scratchpads

We proposed fully integrating RefBank functionality into Scratchpads and had developed a prototype for this integration. However, with the development of ReFinder (see later) and it superseding RefBank’s user interface as the normal mean by which an end user accesses RefBank’s references, this work was placed on hold. Thoughts on how best to progress the integration of ReFinder and Scratchpads will be progressed after the end of the ViBRANT project by Plazi and Pensoft for RefBank/ReFinder and by NHM for Scratchpads.

D7.2 - Mark-up modules
Deliverable report

This product is realised as GoldenGATE Web Services (GgWS), accessible through OBOE. For sustainability, two decisions were made.


 * The first is that GgWS uses the standard OBOE to Scratchpads response mechanism. This decision means that no special code need be developed nor maintained to enable GgWS with Scratchpads.


 * The second is to use Matlab only to format the responses from OBOE to Scratchpads. This was worked out in a (very) mini-workshop between Neil Caithness of WP5 and David King of WP7. Neil and Simon Rycroft had previously asked for help in writing XML and processing it with XSLTs to transfer and reformat results across the two systems. However, on reviewing the requirements, David identified that all tasks could be completed with MatLab. This had the advantage that Neil, and other members of OeRC who might have to maintain OBOE in the future, know MatLab; they do not know XML and XSLT technologies. David then helped to design the program to deliver the necessary inter-system communication within the limits of MatLab. The solution was successfully implemented by Neil with no further assistance being necessary.

Conclusion
The integration that is immediately necessary to promote uptake of the bibliography and mining services has been established. The former enables researchers to effortlessly contribute to RefBank it they are already using Scratchpads to manage their project's data. the latter, is more of an under the hood enabler, so that researchers can exploit GgWS in their projects without having to establish a communication mechanism; however, the integration of their chosen mark-up service into their workflow will still require effort.

Appendix - history
This milestone was subject to much change and delay. The changes and factors for the changes were recorded in the ViBRANT Scratchpad milestone entry during the project. However, to tidy up that milestone page the history there has been reduced the one sentence summary under Rescheduling and the full account attached to this report as this appendix.

Change of date

The original DoW submitted in 2010 gave a target date of 31 March 2013 for this milestone. Given the rescheduling of Scratchpads 2.0 and the architectural changes approved in Man Comm 8 it is more realistic to move the date back to 31 May 2013.

The new target date allows for the inclusion of the new ReFinder service in the integration.

The target does not affect existing integration work which has already seen the successful searching of RefBank from Scratchpads and the successful return of output from GoldenGATE web services run through OBOE.

Further Changes of date

Following a meeting between WP2 and WP7 at the OU on Monday 29 April 2013 a plan was agreed to complete the full bi-directional integration of RefBank and Scratchpads, achievable by then target date of 31 May 2013 for this milestone.

Subsequent follow up with Pensoft, now developing ReFinder the key component in D7.3 and whose search facility is also to be integrated into Scratchpads as part of this milestone indicates that the preliminary interface will not be ready in time. See file, http://vbrant.eu/sites/vbrant.eu/files/ReFinder_status_03May2013.txt ReFinder_Status_03May2013. The full ReFinder interface will be due as part of the deliverable itself in October 2013.

Therefore, I have decided to defer this milestone to the end of July so that progress against both RefBank and ReFinder can be recorded fully. Further interim updates will be made on this page to track progress.

Tuesday 30 July 2013

On review, following my return from holiday, the current status is that:


 * full integration Scratchpads&rarr;RefBank has been achieved with a standalone service delivering this functionality ready for deployment
 * work on a raw RefBank&rarr;Scratchpads has stalled owing to the developer being re-allocated to other OU work but should be resumed in August
 * work on ReFinder&rarr;Scratchpads has stalled pending the release of BDJ and the availability of Pensoft personnel to complete development

Friday 27 September 2013

Reviewing the state of affairs after the hiatus of summer holidays and BIH2013, shows that little progress has been made with the last integration task, from Scratchpads to RefBank, owing to people being away. In the same time period though, progress with ReFinder supersedes this task. ReFinder will now serve as the primary means of accessing the bibliography of life, with RefBank acting as the repository. This change in usage patterns forces a re-think of the milestone. The opportunity afforded by all the required parties being present at pro-iBiosphere's workshops in Berlin during the second week of October will give us the opportunity to complete that rethinking. A further update will be produced following those discussions. Crucially, these discussions and re-scheduling of this milestone will not affect delivery of D7.3, of which this milestone forms a part.

Friday 25 October 2013

Reviewing the state of affairs after another month and found that there has been a breakdown in communications over the deployment of the Scratchpads-to-RefBank harvester. In addition, the preferred hosting of the harvester service on the Scratchpads server is not viable. Therefore, alternatives will have to be explored, preferably one or another of the existing RefBank nodes. This should be completed by mid-November - next week's TDWG conference holding things up for one week. Note: as mentioned above, this delay has not affected delivery of the final deliverable D7.3, which was marked as completed yesterday, Thursday 23 October.

Thursday 14 November 2013

Generously PLAZI has agreed to host the outstanding component for this milestone, the Harvester, but want some changes to it before deployment. The changes will be made by the end of this month. In the meantime, the milestone report for the other components will be completed now.