Web Services

The GoMSS Nowcast project relies on a collection of web services to provide version control repository hosting, issue tracking, documentation rendering and hosting, and software package distribution. Together this collection of interlinked web service accounts provide a web portal for the project.

This section described how those web service accounts are set up, managed, and interconnected.

Software Repositories and Issue Tracking

A team account on the bitbucket.org service is used to provide centralized, publicly accessible storage for the Mercurial version control repositories that contain the project’s code and documentation. Bitbucket is also used to provide an issue tracker for each repository.

The Bitbucket Cloud Teams page provides information on the concept of Bitbucket Teams.

A base account for the project with the user name gomss-nowcast-project was created from the Bitbucket Signup page. The credentials for that account are controlled by Doug Latornell <doug.latornell@43ravens.ca>.

Using the gomss-nowcast-project account, a team called GoMSS-Nowcast was created with the team id gomss-nowcast. The team’s overview page is at https://bitbucket.org/gomss-nowcast/.

User Groups Management

The gomss-nowcast-project account defaults to being a member of both the Administrators and the Developers groups of the gomss-nowcast team. The Manage Teams > User groups page was used to add Doug Latornell to the Administrators and Developers groups.

Members of the Administrators can manage what groups other users are members of, and what type of access (Read, Write, or Admin) they have to each of the team repositories.

Members of the Developers group have write access to all team repositories, and can create new team repositories.

Repository Issue Trackers

When a new repository is created there is a Issue tracker checkbox that can be clicked to provide the repository with an Issue Tracker. See https://bitbucket.org/gomss-nowcast/docs/issues for an example.

If an issue tracker was not enabled when a repository was created it can be added later on the repository’s Settings > Issue tracker settings page.

There are also Settings > Issues links to set up lists of Components, Milestones, and Versions that can be used to classify and organize issues.

Repository issue trackers can be used to organizer development tasks, bug reports, and enhancement ideas.

Documentation Rendering and Hosting

An account on the readthedocs.org service is used to render and host the main project documentation as well as documentation from project software packages.

An account for the project with the user name gomss-nowcast was created from the readthedocs.org Signup page. The credentials for that account are controlled by Doug Latornell <doug.latornell@43ravens.ca>.

To render and host documentation on readthedocs.org from a version control repository the repository must be imported into a readthedocs.org project. Repositories can only be set up a projects from the gomss-nowcast account. However, once a project has been created other users’ accounts on readthedocs.org can be linked as administrators into the project.

Project administrators have access to:

  • An admin interface that provides control over various aspects of the rendering and hosting, including deletion of the project
  • A button that can be used to force a build of the project
  • The docs build history for the project, enabling them to debug build problems

Project Set-up

To set up a readthedocs.org project to render and host the documentation from a Bitbucket repository use the Import a Project button on the dashboard page. Then use the Import Manually button to get to the Project Details page.

Fill in the Name text box. The name that you choose will be the sub-domain part of the URL at which the docs will be hosted; e.g. the name gomss-nowcast will produce docs at http://gomss-nowcast.readthedocs.io/. Each project name must be unique.

Fill in the Repository URL text box with the URL of the repository containing the source files for the documentation that you want to render and host in the project; for example, https://bitbucket.org/gomss-nowcast/docs.

Set the Repository type selector to Mercurial.

Click the Next button.

If all goes well, the repository will be cloned from Bitbucket on to a readthedocs server, Sphinx will be run on the directory tree containing the conf.py file (typically docs/), and the rendered docs will be viewable at https://<project-name>.readthedocs.io/. In other words, an automated version of the process described in Building the Documentation to Preview Changes.

Webhook to Build Docs When Changes are Pushed

To cause the documentation hosted on readthedocs to be automatically updated every time changes are pushed to the corresponding repository on Bitbucket use the Settings > Webhooks page of the repository on Bitbucket.

Click the Add webhook button.

Fill in the Title text box with readthedocs.

Fill in the URL text box with http://readthedocs.org/build/<project_name>, where <project_name> is the project name that you chose for this repository on readthedocs.

Click the Save button.

To test the webhook, make a change to a file in your local clone of the repository, commit the change, and push it to Bitbucket.

  • On the Settings > Webhooks > View requests page for the readthedocs webhook you created on Bitbucket you should see a record of the webhook request being sent to http://readthedocs.org/build/<project_name>.
  • On the Projects > <project_name> > Builds page for the readthedocs project associated with the repository you should see a build in progress or recently completed.

Software Distribution Channel

An anaconda.org organization is used to host and distribute Python packages for installation via the conda package manager.

A personal anaconda.org account is required in order to create an organization. The GoMSS Nowcast organization was created by Doug Latornell <doug.latornell@43ravens.ca>. Other personal anaconda.org accounts can be given write or administrative access to the GoMSS Nowcast organization. Write access to the organization allows users to upload conda packages. Administrative access provides full control over all packages and users in the organization including the ability to add/delete users and groups of users.

Installing Packages from the Channel

To use conda packages you need to have installed the Anaconda Python distribution or the Miniconda package manager. Please see the conda docs for an introduction to conda, and advice on which installation option to choose.

Regardless of which installation option you choose, Python 3 is strongly recommended. All of the packages in the GoMSS Nowcast organization channel are developed, tested, and packaged for Python 3.5.

To install a package from the GoMSS Nowcast organization channel use the -c gomss-nowcast option in the conda install command, for example:

$ conda install -c gomss-nowcast nemo_nowcast

You can also add gomss-nowcast to the channels section of your .condarc file to cause the GoMSS Nowcast organization channel to be searched every time you use the conda install command; see the conda configuration docs for details.

To use the GoMSS Nowcast organization channel in a conda environment description file, include a stanza like:

channels:
  - gomss-nowcast
  - defaults