x/swiftonhpss
Code Issues Proposed changes
swiftonhpss/doc/asciidoc/swiftonhpss_deploy.txt
Phil Bridges 9608a30508 Update documentation for HPSS client API integration 
A few things have changed, so I'm touching up the docs before
I leave.

Change-Id: I0b860fa9a8819c4c61922faa8601e3c72d237ee3
2018-07-03 13:09:04 -05:00

372 lines
10 KiB
Plaintext
Raw Permalink Blame History

SwiftOnHPSS Preliminary Deployment Guide
========================================
Phil Bridges <pgbridge@us.ibm.com>
:Author Initials: PGB
:toc:
:icons:
:numbered:
:website: http://www.hpss-collaboration.org
[red]*Disclaimer: This interface is not certified for use in an HPSS production
environment to store/retrieve production (valued) data! +
It has not been extensively tested, and is still early in development. +
Only for use on a dedicated SwiftOnHPSS development system!*
This document is meant to provide general instruction on the installation and
configuration of a stand-alone Swift node running SwiftOnHPSS.
[[X1]]
Working Assumptions
-------------------
This guide assumes a good working knowledge of:
* OpenStack Swift
* OpenStack Keystone
* HPSS administration
* Basic Linux system administration
This guide also makes assumptions on your:
* Client machine: running RHEL 7.1 on a reasonably modern x86-64 processor,
with Internet access for the duration of the install and LAN connectivity
to the HPSS core and movers at all times.
* HPSS installation: HPSS 7.5 or newer, using UNIX authentication with
local or system passwd files
* OpenStack infrastructure: that there exists a Keystone server already,
configured with a service tenant and 'swift' service account.
[[X2]]
Initial Considerations
----------------------
Install the HPSS client. This machine will run a stand-alone
Swift node with SwiftOnHPSS.
Add a 'swift' service user in HPSS and on the client machine:
----
# adduser swift
# /opt/hpss/bin/hpss_unix_user add swift
----
Also, make sure that the /swift folder in HPSS is owned by the 'swift' service
user.
----
# /opt/hpss/bin/scrub
scrub> chown /swift swift recurse top
scrub> quit
----
[[X3]]
Install Prerequisites for SwiftOnHPSS
-------------------------------------
It's recommended to install 'pip', which is a Python package management tool.
That can be found at https://pip.pypa.io:
----
# wget https://pip.pypa.io/get-pip.py
# python get-pip.py
----
Install memcached, start the service and make it run on startup.
----
# yum install memcached python-memcached
# service memcached start
# systemctl enable memcached
----
[[X4]]
Install Keystone (if not already existing)
------------------------------------------
Install OpenStack Keystone, and deploy it following
this guide: https://docs.openstack.org/kilo/install-guide/yum/content/keystone-install.html
This does not have to be on the same machine that SwiftOnHPSS is on, but it does
need to be installed on an HPSS client machine if the HPSS Keystone driver is
desired to use HPSS identities with Swift.
After installing Keystone and verifying that it works using that guide,
create the service entity and API endpoint, but don't create any users or
projects yet if the HPSS-specific Keystone backend is desired.
If it is desired, install the HPSS Keystone backend, and configure
/etc/keystone/keystone.conf to use it as the sole identity provider.
----
# cd /opt/openstack
# git clone https://github.com/hpss-collaboration/hpss-keystone-driver
# cd hpss-keystone-driver
# pip install -r requirements.txt
# python setup.py develop
----
./etc/keystone/keystone.conf
----
[identity]
driver = hpss_keystone.identity.Identity
----
[[X5]]
Install and Configure SwiftOnHPSS
---------------------------------
We have tested SwiftOnHPSS against the Liberty stable release of Swift.
Download the Liberty release of Swift (https://launchpad.net/swift/liberty/2.5.0, or
more optimally from https://github.com/openstack/swift) and install it.
Additionally, install the 'keystonemiddleware' library using Pip if you are
using Keystone.
----
# mkdir /opt/openstack
# cd /opt/openstack
# git clone https://github.com/openstack/swift stable/liberty
# git clone https://github.com/openstack/keystone stable/liberty
# git clone https://github.com/hpss-collaboration/swiftonhpss
# pip install keystonemiddleware
# pip install -r swift/requirements.txt
# pip install -r swift/setup_requirements.txt
# pip install -r keystone/requirements.txt
# cd swiftonhpss
# python setup.py develop
----
Contact your HPSS support representative for the 'hpss' Python library
appropriate for your site's version of HPSS.
Upon receipt of those libraries, install them. Assuming the libraries are in
/opt/hpss_lib:
----
# cd /opt/hpss_lib
# python hpssfs/setup.py install
# python hpsspy/setup.py install
# pip install -r swiftonhpss/prereqs.txt
# python swiftonhpss/setup.py develop
----
Copy the sample configuration files over from the 'etc' folder of the Swift
installation over to /etc/swift, and remove the extra account and object
server configurations.
----
# cp -r doc/saio/etc/ /etc/swift
# rm /etc/swift/account-server/2.conf
# rm /etc/swift/account-server/3.conf
# rm /etc/swift/account-server/4.conf
# rm /etc/swift/container-server/2.conf
# rm /etc/swift/container-server/3.conf
# rm /etc/swift/container-server/4.conf
# rm /etc/swift/object-server/2.conf
# rm /etc/swift/object-server/3.conf
# rm /etc/swift/object-server/4.conf
----
Then go through the object server config for SwiftOnHPSS and alter it like this:
./etc/swift/object-server/1.conf:
[source,conf]
----
[DEFAULT]
hpss_swift_dir = /swift
mount_check = false
disable_fallocate = true
bind_ip = 127.0.0.1
bind_port = 6010
workers = 1
user = swift
log_facility = LOG_LOCAL2
recon_cache_path = /var/cache/swift
eventlet_debug = true
[pipeline:main]
pipeline = recon object-server
[app:object-server]
use = egg:swiftonhpss#object
[filter:recon]
use = egg:swift#recon
### Specifically comment these out, because we don't want to use any of these
### extra servers.
#[object-replicator]
#vm_test_mode = yes
#[object-reconstructor]
#[object-updater]
#[object-auditor]
----
./etc/swift/proxy-server.conf:
[source,conf]
----
[DEFAULT]
bind_ip = 127.0.0.1 # Replace this with your node's IP address!
bind_port = 8080
workers = 1
user = swift
log_facility = LOG_LOCAL1
eventlet_debug = true
[pipeline:main]
# Yes, proxy-logging appears twice. This is so that
# middleware-originated requests get logged too.
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache bulk tempurl ratelimit crossdomain authtoken keystoneauth staticweb container-quotas account-quotas slo dlo proxy-logging proxy-server
[filter:catch_errors]
use = egg:swift#catch_errors
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:proxy-logging]
use = egg:swift#proxy_logging
[filter:bulk]
use = egg:swift#bulk
[filter:ratelimit]
use = egg:swift#ratelimit
[filter:crossdomain]
use = egg:swift#crossdomain
[filter:dlo]
use = egg:swift#dlo
[filter:slo]
use = egg:swift#slo
[filter:tempurl]
use = egg:swift#tempurl
#[filter:tempauth]
#use = egg:swift#tempauth
#user_admin_admin = admin .admin .reseller_admin
#user_test_tester = testing .admin
#user_test2_tester2 = testing2 .admin
#user_test_tester3 = testing3
### Replace all of this with your Keystone server's configuration!
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
identity_uri = http://127.0.0.1:35357/ # Replace with Keystone server IP
admin_tenant_name = service
admin_user = swift
admin_password = password
auth_uri = http://127.0.0.1:5000/ # Replace with Keystone server IP
#cache = swift.cache
include_service_catalog = False
delay_auth_decision = True
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin, swiftoperator
reseller_prefix = KEY
[filter:staticweb]
use = egg:swift#staticweb
[filter:account-quotas]
use = egg:swift#account_quotas
[filter:container-quotas]
use = egg:swift#container_quotas
[filter:cache]
use = egg:swift#memcache
[filter:gatekeeper]
use = egg:swift#gatekeeper
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
----
After editing the configuration scripts, it is time to generate the Swift
server rings. There is a script to automate doing this in
doc/saio/bin/remakerings that could be used, but it needs some modification
to fit the recommended configuration.
----
# cp doc/saio/bin/remakerings ./remakerings-for-HPSS
# vim remakerings-for-HPSS
----
../remakerings-for-HPSS:
[source,shell]
----
#!/bin/bash
cd /etc/swift
rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
swift-ring-builder object.builder create 0 1 1
swift-ring-builder object.builder add r1z1-127.0.0.1:6010/ 1
swift-ring-builder object.builder rebalance
swift-ring-builder container.builder create 0 1 1
swift-ring-builder container.builder add r1z1-127.0.0.1:6011/metadata 1
swift-ring-builder container.builder rebalance
swift-ring-builder account.builder create 0 1 1
swift-ring-builder account.builder add r1z1-127.0.0.1:6012/metadata 1
swift-ring-builder account.builder rebalance
----
----
# ./remakerings-for-HPSS
----
Next, configure rsyslog for Swift. Swift comes with a perfectly usable
example configuration, so just copy it over.
----
# cp doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/10-swift.conf
# service syslog restart
----
Next, in order to work around a known linking bug, it's necessary to
preload the libtirpc library when starting up the Swift object server, or else
SwiftOnHPSS will hang and then fail to connect to HPSS.
One could just throw LD_PRELOAD=libtirpc.so in front of it and it'll probably work
So, start up the Swift server, and make sure that it is accessible and works
properly.
----
# LD_PRELOAD=libtirpc.so swift-init main start
----
[[X6]]
Where To Go From Here
---------------------
At this point, the SwiftOnHPSS archive machine is fully configured and ready
to develop on and use for experimental purposes.
A command-line client can be had by installing the "python-openstackclient"
package with 'pip'. If Keystone is in use, then in order to log into Swift it
is necessary to create a Swift service user in Keystone and grant it the
'admin' role in the 'service' project in the default domain, and also to add
the Swift URL to the service endpoint catalog.
SwiftOnHPSS now has a tool to synchronize changes made through other HPSS
interfaces with the Swift metadata databases, called 'swiftonhpss-nstool'.
It is included in the SwiftOnHPSS Git repository, and should have been
automatically installed along with the rest of the system.
For further reading on how the Swift object storage system works and how to do
further customization, see:
https://docs.openstack.org/developer/swift/index.html
View Git Blame Copy Permalink