refstack/doc/source/refstack.rst

===================
RefStack Quickstart
===================

You can use docker for `one-click setup <run_in_docker.html>`__ or follow
step-by-step instructions below. These instructions have been tested on
Ubuntu 14 and 16 LTS.

Install API dependencies
^^^^^^^^^^^^^^^^^^^^^^^^
::

   sudo apt-get install git python-dev python-virtualenv libssl-dev build-essential libffi-dev
   sudo apt-get install mysql-server python-mysqldb

Install RefStack UI dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
   curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
   echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
   sudo apt-get update && sudo apt-get install -y nodejs yarn

Setup the RefStack database
^^^^^^^^^^^^^^^^^^^^^^^^^^^

**Log into MySQL**::

   mysql -u root -p

**After authentication, create the database**::

   CREATE DATABASE refstack;

**Create a refstack user**::

   CREATE USER 'refstack'@'localhost' IDENTIFIED BY '<your password>';

**or using hash value for your password**::

   CREATE USER 'refstack'@'localhost' IDENTIFIED BY PASSWORD '<hash value of your password';

**Grant privileges**::

   GRANT ALL PRIVILEGES ON refstack . * TO 'refstack'@'localhost';

**Reload privileges**::

   FLUSH PRIVILEGES;

**Exit MySQL**::

   quit

Clone the repository
^^^^^^^^^^^^^^^^^^^^
::

   git clone https://opendev.org/openinfra/refstack
   cd refstack

**Create virtual environment**::

   virtualenv .venv --system-site-package

**Source to virtual environment**::

   source .venv/bin/activate

**Update pip**::

   pip install -U pip

**Install environment pre-requirements**::

   pip install gunicorn==18 # App server
   pip install PyMySQL>=0.6.2,!=0.6.4 # python mysql connector

Install RefStack application
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   pip install .

Install needed RefStack UI library dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   yarn

API configuration file preparation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Make a copy of the sample config file (etc/refstack.conf.sample) and
update it with the correct information of your environment. Examples
of the config parameters with default values are included in the
sample config file.

You should ensure that the following values in the config file are
noted and properly set:

``connection`` field in the ``[database]``\ section.

For example, if the backend database is MySQL then update:

``#connection = <None>`` to
``connection = mysql+pymysql://refstack:<your password>@x.x.x.x/refstack``

``ui_url`` field in the ``[DEFAULT]`` section.

   This should be the URL that the UI can be accessed from. This will
   likely be in the form ``http://<your server IP>:8000`` (8000 being
   the default port RefStack is hosted on). For example:

   ``http://192.168.56.101:8000``

``api_url`` field in the ``[api]`` section.

   This should be the URL that the API can be accessed from. This, in
   most cases, will be the same as the value for ``ui_url`` above.

``app_dev_mode`` field in the ``[api]`` section.

   Set this field to true if you aren't creating a production-level
   RefStack deployment and are just trying things out or developing.
   Setting this field to true will allow you to quickly bring up both
   the API and UI together, with the UI files being served by a simple
   file server that comes with Pecan.

Create UI config file
^^^^^^^^^^^^^^^^^^^^^

From the RefStack project root directory, create a config.json file and
specify your API endpoint inside this file. This will be something like
{"refstackApiUrl": "http://192.168.56.101:8000/v1"}::

   cp refstack-ui/app/config.json.sample refstack-ui/app/config.json

Openstack OpenID endpoint configuration (optional)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you are only interested in the uploading and viewing of result sets,
then this section can be ignored. However, in order for user accounts
and authentication to work, you need to make sure you are properly
configured with an OpenStack OpenID endpoint. There are two options:

-  Use the official endpoint
   `openstackid.org <https://openstackid.org>`__
-  Host your own openstackid endpoint

Since openstackid checks for valid top-level domains, in both options
you will likely have to edit the hosts file of the system where your
web-browser for viewing the RefStack site resides. On Linux systems, you
would modify ``/etc/hosts``, adding a line like the following:

``<RefStack server IP>  <some valid domain name>``

Example:

``192.168.56.101  myrefstack.com``

On Windows, you would do the same in
``%SystemRoot%\System32\drivers\etc\hosts``. Alternatively, you can add
a custom DNS record with the domain name mapping if possible.

Note that doing this requires you to modify the config.json file and the
``api_url`` and ``ui_url`` fields in refstack.conf to use this domain
name instead of the IP.

**Option 1 - Use Official Endpoint**

Using the official site is probably the easiest option as no additional
configuration is needed besides the hosts file modifications as noted
above. RefStack, by default, points to this endpoint.

**Option 2 - Use Local Endpoint**

Instructions for setting this up are outside of the scope of this doc,
but you can get started at
`Openstackid project <https://opendev.org/openinfra/openstackid>`__ .
You would then need to modify the ``openstack_openid_endpoint`` field in
the ``[osid]`` section in refstack.conf to match the local endpoint.

Database sync
^^^^^^^^^^^^^

**Check current revision**::

   refstack-manage --config-file /path/to/refstack.conf version

The response will show the current database revision. If the revision is
``None`` (indicating a clear database), the following command should be
performed to upgrade the database to the latest revision:

**Upgrade database to latest revision**::

   refstack-manage --config-file /path/to/refstack.conf upgrade --revision head

**Check current revision**::

   refstack-manage --config-file /path/to/refstack.conf version

::

    Now it should be some revision number other than `None`.

(Optional) Generate About Page Content
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The RefStack About page is populated with HTML templates generated from
our RST documentation files. If you want this information displayed, then
run the following command from the root of the project.

::

   ./tools/convert-docs.py -o ./refstack-ui/app/components/about/templates ./doc/source/*.rst

Ignore any unknown directive errors.

Start RefStack
^^^^^^^^^^^^^^

A simple way to start refstack is to just kick off gunicorn using the
``refstack-api`` executable::

   refstack-api --env REFSTACK_OSLO_CONFIG=/path/to/refstack.conf

If ``app_dev_mode`` is set to true, this will launch both the UI and
API.

Now available:

-  ``http://<your server IP>:8000/v1/results`` with response JSON
   including records consisting of ``<test run id>`` and
   ``<upload date>`` of the test runs. The default response is limited
   to one page of the most recent uploaded test run records. The number
   of records per page is configurable via the RefStack configuration
   file. Filtering parameters such as page, start\_date, and end\_date
   can also be used to specify the desired records. For example: GET
   ``http://<your server IP>:8000/v1/results?page=n`` will return page
   *n* of the data.

-  ``http://<your server IP>:8000/v1/results/<test run id>`` with
   response JSON including the detail test results of the specified
   ``<test run id>``

(Optional) Configure Foundation organization and group
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Overall RefStack admin access is given to users belonging to a
"Foundation" organization. To become a Foundation admin, first a
"Foundation" organization must be created. Note that you must have
logged into RefStack at least once so that a user record for your
account is created.

**Log into MySQL**::

   mysql -u root -p

**Create a group for the "Foundation" organization**::

   INSERT INTO refstack.group (id, name, created_at) VALUES (UUID(), 'Foundation Group', NOW());

**Get the group ID for the group you just created**::

   SELECT id from refstack.group WHERE name = 'Foundation Group';

**Get your OpenID**::

   SELECT openid from refstack.user WHERE email = '<your email>';

**Add your user account to the previously created "Foundation" group.**

Replace ``<Group ID>`` and ``<Your OpenID>`` with the values
retrieved in the two previous steps::

   INSERT INTO refstack.user_to_group (created_by_user, user_openid, group_id, created_at)    VALUES ('<Your OpenID>', '<Your OpenID>', '<Group ID>', NOW());

**Create the actual "Foundation" organization using this group**::

   INSERT INTO refstack.organization (id, type, name, group_id, created_by_user, created_at)    VALUES (UUID(), 0, 'Foundation', '<Group ID>', '<Your OpenID>', NOW());

(Optional) Build documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The RefStack documentation can be build using following commands::

   cd ~/refstack; source .venv/bin/activate
   sudo apt-get install python3-dev python-tox
   tox -e docs

The documentation files will be build under ``~/refstack/build/sphinx``.