x/swiftonhpss
Code Issues Proposed changes

Merge pull request #21 from pushpesh/quickstartguide

Browse Source 
Quickstartguide
This commit is contained in:
Thiago da Silva 2014-07-31 10:10:10 -04:00
commit 8a7e143127
1 changed files with 92 additions and 200 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

292
doc/markdown/quick_start_guide.md
View File

@ -3,247 +3,134 @@
## Contents
* [Overview](#overview)
* [System Setup](#system_setup)
* [Gluster For Swift Setup](#swift_setup)
* [Using Gluster for Swift](#using_swift)
* [SwiftOnFile Setup](#swift_setup)
* [Using SwiftOnFile](#using_swift)
* [What now?](#what_now)
<a name="overview" />
## Overview
Gluster-swift allows GlusterFS to be used as the backend to the object
store OpenStack Swift.
SwiftOnFile allows any POSIX complaint filesystem (which supports extended attributes) to be used as the backend to the OpenStack Swift(Object Store).
The following guide will get you quickly started with a gluster-swift
environment on a Fedora or RHEL/CentOS system. This guide is a
great way to begin using gluster-swift, and can be easily deployed on
a single virtual machine. The final result will be a single gluster-swift
node.
The following guide assumes you have a running [OpenStack Swift SAIO setup][], and you want to extend this setup to try SwiftOnFile as Storage Policy on a with XFS partition or GlusterFS volume. This will get you quickly started with a SwiftOnFile deployment on a Fedora or RHEL/CentOS system.
> NOTE: In Gluster-Swift, accounts must be GlusterFS volumes.
This guide will not go on detail on how to prepare a Swift SAIO setup or how to create a gluster volume (or other FS).This guide assumes you know about these technologies, if you require any help in setting those please refer to the links provided.
<a name="system_setup" />
## System Setup
### Prerequisites on CentOS/RHEL
On CentOS/RHEL you will need to setup GlusterFS and EPEL repos.
#### GlusterFS CentOS/RHEL Repo
1. OpenStack SAIO deployment(this guide uses SAIO on Fedora20)
2. One xfs partition /glusterfs volume - named vol
* CentOS
>Note: Swift SAIO deployment should contain Storage Policy code changes. SwiftOnFile should work with OpenStack Swift2.0.0 release and onwards. Icehouse and prior release of OpenStack Swift does not have Storage Policy code.
Each xfs partition/glusterfs volume will be defined as a separate storage policy.
### Install SwiftOnfile
1. Before you begin swiftonfile setup please ensure you have OpenStack Swift SAIO setup up & running. Please refer to the SAIO guide for this.
2. cd $HOME; git clone https://github.com/swiftonfile/swiftonfile.git
3. cd $HOME/swiftonfile;python setup.py develop;cd $HOME
### Configure SwiftOnFile as Storage Policy
#### Object Server Configuration
An SAIO setup emulates a four node swift setup and should have four object server running. Add another object server for SwiftOnFile DiskFile API implementation by setting the following configurations in the file /etc/swift/object-server/5.conf:
~~~
wget -O /etc/yum.repos.d/glusterfs-epel.repo \
http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
~~~
[DEFAULT]
devices = /mnt/xfsvols/
mount_check = false
bind_port = 6050
max_clients = 1024
workers = 1
disable_fallocate = true
* RHEL
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:gluster_swift#object
user = root
log_facility = LOG_LOCAL2
log_level = DEBUG
log_requests = on
disk_chunk_size = 65536
~~~
>Note: The parameter 'devices' tells about the path where your xfs partition or glusterfs volume is mounted. The sub directory under which your xfs partition or glusterfs volume is mounted will be called device name.
>For example: You have a xfs formated partition /dev/sdb1, and you mounted it under /mnt/xfsvols/vol, then your device name would be 'vol' & and the parameter 'devices' would contain value '/mnt/xfsvols'.
#### Setting SwiftOnFile as storage policy
Edit /etc/swift.conf to add swiftonfile as a storage policy:
~~~
wget -O /etc/yum.repos.d/glusterfs-epel.repo \
http://download.gluster.org/pub/gluster/glusterfs/LATEST/RHEL/glusterfs-epel.repo
[storage-policy:0]
name = swift
default = yes
[storage-policy:1]
name = swiftonfile-test
~~~
You can also make "swiftonfile-test" the default storage policy by using the 'default' parameter.
#### EPEL CentOS/RHEL Repo
Please refer to [EPEL][] for more information on how to setup the EPEL repo.
### Required Package Installation
Install and start the required packages on your system to create a GlusterFS volume.
#### Prepare rings
Edit the remakerings script to prepare rings for this new storage policy:
~~~
yum install glusterfs glusterfs-server glusterfs-fuse memcached xfsprogs
swift-ring-builder object-1.builder create 1 1 1
swift-ring-builder object-1.builder add r1z1-127.0.0.1:6050/vol 1
swift-ring-builder object-1.builder rebalance
~~~
Execute the remakerings script to prepare new rings files.
In a SAIO setup remakerings scipt is usually situated at ~/bin/remakerings.You can also run above rings builder commands manually.
#### Start services
Notice the mapping between SP index (1) defined in conf file above and the object ring builder command.
Type the following to start `memcached` and `glusterfs` services:
#### Load the new configurations
Restart swift services to reflect new changes:
~~~
service memcached start
service glusterd start
swift-init main restart
~~~
Type the following to start the services automatically on system startup:
~~~
chkconfig memcached on
chkconfig glusterd on
~~~
### Gluster Volume Setup
Now you need to determine whether you are going to use a partition or a loopback device
for storage.
#### Partition Storage Setup
If you are using a separate disk partition, please execute the following instructions
to create a GlusterFS brick:
~~~
mkfs.xfs -i size=512 /dev/<disk partition>
mkdir -p /export/brick
~~~
Add the following line to `/etc/fstab` to mount the storage automatically on system
startup:
~~~
/dev/<disk partition> /export/brick xfs inode64,noatime,nodiratime 0 0
~~~
Now type the following to mount the storage:
~~~
mount -a
~~~
#### Loopback Storage Setup
If you do not have a separate partition, please execute the following instructions
to create a disk image as a file:
~~~
truncate -s 5GB /srv/swift-disk
mkfs.xfs -i size=512 /srv/swift-disk
mkdir -p /export/brick
~~~
Add the following line to `/etc/fstab` to mount the storage automatically on system
startup:
~~~
/srv/swift-disk /export/brick xfs loop,inode64,noatime,nodiratime 0 0
~~~
Now type the following to mount the storage:
~~~
mount -a
~~~
### Create a GlusterFS Volume
You now need to create a GlusterFS volume (make sure your hostname is in /etc/hosts or is DNS-resolvable)
~~~
mkdir /export/brick/b1
gluster volume create myvolume `hostname`:/export/brick/b1
gluster volume start myvolume
~~~
<a name="swift_setup" />
## Gluster-Swift Setup
### Repository Setup on RHEL/CentOS
Gluster-Swift requires OpenStack Swift's Havana release, which
may not be available on some older operating systems. For RHEL/CentOS
systems, please setup Red Hat RDO's repo by executing the following command:
~~~
yum install -y http://rdo.fedorapeople.org/rdo-release.rpm
~~~
### Download
Download the latest Havana release RPMs from [launchpad.net downloads][]:
### Install
Install the RPM by executing the following:
~~~
yum install -y <path to RPM>
~~~
### Enabling gluster-swift accross reboots
Type the following to make sure gluster-swift is enabled at
system startup:
~~~
chkconfig openstack-swift-proxy on
chkconfig openstack-swift-account on
chkconfig openstack-swift-container on
chkconfig openstack-swift-object on
~~~
#### Fedora 19 Adjustment
Currently gluster-swift requires its processes to be run as `root`. You need to
edit the `openstack-swift-*.service` files in
`/etc/systemd/system/multi-user.target.wants` and change the `User` entry value
to `root`.
Then run the following command to reload the configuration:
~~~
systemctl --system daemon-reload
~~~
### Configuration
As with OpenStack Swift, gluster-swift uses `/etc/swift` as the
directory containing the configuration files. You will need to base
the configuration files on the template files provided. On new
installations, the simplest way is to copy the `*.conf-gluster`
files to `*.conf` files as follows:
~~~
cd /etc/swift
for tmpl in *.conf-gluster ; do cp ${tmpl} ${tmpl%.*}.conf; done
~~~
#### Generate Ring Files
You now need to generate the ring files, which inform gluster-swift
which GlusterFS volumes are accessible over the object
storage interface. The format is
~~~
gluster-swift-gen-builders [VOLUME] [VOLUME...]
~~~
Where *VOLUME* is the name of the GlusterFS volume which you would
like to access over gluster-swift.
Let's now expose the GlusterFS volume called `myvolume` you created above
by executing the following command:
~~~
cd /etc/swift
/usr/bin/gluster-swift-gen-builders myvolume
~~~
### Start gluster-swift
Use the following commands to start gluster-swift:
~~~
service openstack-swift-object start
service openstack-swift-container start
service openstack-swift-account start
service openstack-swift-proxy start
~~~
<a name="using_swift" />
## Using gluster-swift
### Create a container
#### Running functional tests
TBD
## Using SwiftOnFile
It is assumed that you are still using 'tempauth' as authentication method, which is default in SAIO deployment.
#### Get the token
~~~
curl -v -H 'X-Auth-User: test:tester' -H "X-Auth-key: testing" -k http://localhost:8080/auth/v1.0
~~~
Use 'X-Auth-Token' & 'X-Storage-Url' returned in above request for all sucequent request.
#### Create a container
Create a container using the following command:
~~~
curl -v -X PUT http://localhost:8080/v1/AUTH_myvolume/mycontainer
curl -v -X PUT -H 'X-Auth-Token: AUTH_XXXX' -H 'X-Storage-Policy: swiftonfile-test' http://localhost:8080/v1/AUTH_test/mycontainer
~~~
It should return `HTTP/1.1 201 Created` on a successful creation. You can
also confirm that the container has been created by inspecting the GlusterFS
volume:
~~~
ls /mnt/gluster-object/myvolume
~~~
It should return `HTTP/1.1 201 Created` on a successful creation.
#### Create an object
You can now place an object in the container you have just created:
~~~
echo "Hello World" > mytestfile
curl -v -X PUT -T mytestfile http://localhost:8080/v1/AUTH_myvolume/mycontainer/mytestfile
curl -v -X PUT -T mytestfile 'X-Auth-Token: AUTH_XXXX' http://localhost:8080/v1/AUTH_test/mycontainer/mytestfile
~~~
To confirm that the object has been written correctly, you can compare the
test file with the object you created:
~~~
cat /mnt/gluster-object/myvolume/mycontainer/mytestfile
cat /mnt/xfsvols/vol/AUTH_test/mycontainer/mytestfile
~~~
#### Request the object
@ -251,22 +138,27 @@ Now you can retreive the object and inspect its contents using the
following commands:
~~~
curl -v -X GET -o newfile http://localhost:8080/v1/AUTH_myvolume/mycontainer/mytestfile
curl -v -X GET -o newfile http://localhost:8080/v1/AUTH_test/mycontainer/mytestfile
cat newfile
~~~
You can also use etag information provided while you do HEAD on object
and compare it with md5sum of the file on your FS.
<a name="what_now" />
## What now?
You now have a single node SwiftOnFile setup ready, next sane step is a multinode swift and SwiftOnFile setup. It is recomended to have a look at [OpenStack Swift deployment guide][] & [Multiple Server Swift Installation][].If you now consider yourself familiar with a typical 4-5 node swift setup, you are good to extent this setup further and add SwiftOnFile DiskFile implementation as a Storage Policy to it. If you want to use SwiftOnFile on a gluster volume, it would be good to have a seprate gluster cluster. We would love to hear about any deployment scenarios involving SOF.
For more information, please visit the following links:
* [Authentication Services Start Guide][]
* [OpenStack Swift deployment guide][]
* [Multiple Server Swift Installation][]
* [OpenStack Swift Storage Policy][]
* [GlusterFS Quick Start Guide][]
* [OpenStack Swift API][]
[GlusterFS Quick Start Guide]: http://www.gluster.org/community/documentation/index.php/QuickStart
[OpenStack Swift API]: http://docs.openstack.org/api/openstack-object-storage/1.0/content/
[Jenkins]: http://jenkins-ci.org
[Authentication Services Start Guide]: auth_guide.md
[EPEL]: https://fedoraproject.org/wiki/EPEL
[launchpad.net downloads]: http://launchpad.net/gluster-swift/havana/1.10.0-2
[OpenStack Swift Storage Policy]: http://docs.openstack.org/developer/swift/overview_policies.html
[OpenStack Swift SAIO setup]: http://docs.openstack.org/developer/swift/development_saio.html
[OpenStack Swift deployment guide]: http://docs.openstack.org/developer/swift/deployment_guide.html
[Multiple Server Swift Installation]: http://docs.openstack.org/developer/swift/howto_installmultinode.html