This allows developers to easily insert the python
debugger to the OpenStack Swift code which in turn
calls gluster-swift.
Using 'git submodule' allows developers to not only create
a clone of the OpenStack Swift repo locally, but
also automatically set the repo to the appropriate tested version.
This change also allows us to always match the version
of OpenStack Swift that gluster-swift has been tested against.
For this change, the submodule of OpenStack Swift has
been set to version 1.9.1 which, according to the RDO
rpms, it is commit id 95bcd7180c546f91414c2493d343fe1a4ae3ce44.
Here is how developers can setup their repo after this change:
$ git submodule init
$ git submodule update
Now that the correct version of the OpenStack Swift
code has been cloned to the modules directory, it can be
used for debugging during development. First we need
to setup a link called 'swift' to point to the python
code from the root of the gluster-swift repo as follows:
$ ln -s swift modules/swift/swift
Here is simple example of how to debug an object PUT command.
First, we need to add the following line:
`import pdb ; pdb.set_trace`
to swift/proxy/controllers/obj.py at line 818.
To execute the debug line added above, we can use the following
unit test:
$ nosetests -v -s -exe test/unit/proxy/test_server.py
Change-Id: I01dd58954e5ef139208a1132140c6c83ada82c18
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5787
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Tested-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
These are a copy of the OpenStack Swift proxy controller
unit tests as of version their 1.9.1.
Change-Id: Ib05bc2a37dbb29d729346d78ea8f6de83b82375b
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5735
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Added a fake_file class that implements minimal set of functions that
are invoked by the code in GET.
BUG: 987841
https://bugzilla.redhat.com/show_bug.cgi?id=987841
Change-Id: I5bdf5be1c0c4c8231f34c9be529e6edc83774f2e
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5511
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The defacto swift implementation has a feature called "fast-POSTs",
which is an optimization that allows POST operations on files to be
performed by updating metadata on additional .meta files. The default
is to treat all POST opreations as a COPY operation. Since this
gluster-swift code stores all metadata directly on the original object
file, there is no need to use an inefficient operation such as COPY.
BUG XXXXXX: (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX)
Change-Id: I39f0a682d5344c3f84c5baa8e292227051c061b8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5673
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
While it is very improbable, it seemed fairly straight forward to add
a small bit of code to ensure the temporary file name is random and
won't clash with another host, pid or thread potentially working on
the same object.
BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX)
Change-Id: I862021b725efbfe58b98754c4470aef4882eb8f7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5671
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
For whatever reason, it appears that GlusterFS, or perhaps FUSE can
continuously return ENOENT on a rename system call even when we have
double checked that there is no reason to do so. That is a bug for
that sub system. However, our response to that bug can result in an
infinite loop, which is bad.
This code reduces that to 10 attempts.
In addition, we restructed the open retry loop to match, providing
module constants for the upper bounds of both retry loops.
BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX)
Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5670
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
When the unit tests are run in an environment like the functional
tests, where the accurrate size in listings switch is on, we need to
sure that the tests that verify the behavior when it is off run in
that state.
Change-Id: I9f3ef90af7ef4edc698e2b8b7f016e6aaa7fac98
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5703
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Moving forward all python scripts in extras/ are to
be checked for pep8 compliancy.
Change-Id: I4f20c1c97590049ba59fd432cbabba89df101e81
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/5714
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
* The master branch is currently tracking the development
version 1.9.1 of Havana OpenStack Swift.
* Seems that we still have some settings which are not
reset in the unit tests. This fix provides a guaranteed
state for test_utils.py:TestUtils.test_get_container_details
Change-Id: Ie32f49743301bfa493f70ed8a539fb6840f9f67b
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5719
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
The code changes are basically:
* Apply refactoring in the DiskFile class to use the new DiskWriter
abstraction
* Move and rename our diskfile module to match upstream
* ThreadPools allow us to remove the tpool usage around fsync
* Update the Ring subclass to support the get_part() method
* Update to use the 1.9.1 proxy server unit tests
* Move the DebugLogger class to test.unit
* Rebuild the Rings to use the new layout
* Remove backup ring builder files
* Update spec files to 1.9.1, and tox to use swift 1.9.1
* Updated version to 1.9.0-0
Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5331
Our initial implementation only required Jenkins
to export binary RPMs, but as we move foward, we really
need to also export SRPMs. To support SRPMs, the spec
file in the RPM has to have the correct NAME, VERSION,
and RELEASE information.
Change-Id: Icd7132b4aafdbe7a1f02a35d0be7ad63b2e7c056
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5669
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-on: http://review.gluster.org/5679
gluster-swift-gen-builders now changes directory to /etc/swift prior to
creating ring files. It does display a message indicating the location of
the ring files.
BUG: 986458
https://bugzilla.redhat.com/show_bug.cgi?id=986458
Change-Id: Iea33c925d98c47c914e8a95d11faa3220967b4d6
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5658
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
* Container and accounts performance increase by removing
the need to update either the object count or the
container count, respectively. New hidden
configuratoins added to re-enable the functionality.
* object_only configuratoin removed from fs.conf and
replaced with a hidden configuration. The new hidden
configuration reports gratuituosly crated directories
as objects, to support a compatibility behavior with
previous version 1.4.8 (even though it was incorrect).
BUG 988969: https://bugzilla.redhat.com/show_bug.cgi?id=988969
Change-Id: Idca20b8629ec38606ff5692fe62bc2cadabffc86
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5403
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-on: http://review.gluster.org/5504
A gluster volume could yield an ENOSPC condition seeing
that a volume is full. This needed to handled correctly.
Added error handling.
BUG: 985253
https://bugzilla.redhat.com/show_bug.cgi?id=985253
Change-Id: I85472c0a81a354a2796327fead606da3a938d4bf
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5362
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Copyrights are updated across the code to reflect 2013.
Copyrights message was added to couple of scripts.
Change-Id: I2cfc7bdc6adaa5c9ab0075aab49021a5f0dea7a1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5371
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
gluster-swift-gen-builders copies the older ring files to
/etc/swift/backup_ring. And then creates the new ring files to /etc/swift
BUG: 986458
Change-Id: I06e93179e46da56d5046419e66c8864aa08f184d
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5384
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Still need to change unit tests to use Havana
Change-Id: I7847e7916043933a47b3553115c79c5d6b28a836
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5312
The overridden configuration files of gluster now disable the usage of
fallocate. Also modified the functional test configuration files
accordingly.
BUG: 951661
Change-Id: I3c7d518e0415b7eea5b31c88dd647069f9d730ac
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5303
Remove a number of stat() system calls that were redundant.
BUG 903396 (https://bugzilla.redhat.com/show_bug.cgi?id=903396)
Change-Id: I2e89ce8a023650cd745f136f5c697902cf2462ea
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5293
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Changed all "we" wording to "you," and altered and rearranged some
wording to help the document flow better when being read.
Change-Id: Idf39eb93666a71e3fda25375080eaae90a23ed52
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5306
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The RPM can now be installed on a system to upgrade previous
installation of gluster-swift and gluster-ufo (which replaced
gluster-swift-plugin)
It also replaces all the glusterfs-swift-* servers
Change-Id: I1c1eeaa1b0a93fdb1eaf2ed018a365c4415c06b0
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5297
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Change-Id: Ibcef5d817390a11d85cf125bb9addfbfded1b019
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5301
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Added instructions on how to amend changes to a previous commit.
Included commands and brief instructions.
Change-Id: I078c9392d71273f559357a014768aada85ad3405
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5295
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Changed "usage" to "user" to match title. Also, prefixed
"pip install" with proper "sudo" command, due to command requiring
super user privelages.
Change-Id: I972bcb96fd476a4022385a418c86916038ec1aab
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5294
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
By storing the functional tests configuration files in the
repo, we can now run the functional_tests.sh to setup,
run the functional tests, and teardown.
Most likely this will be able to be run as a user from
the same directory as the repo, but at the moment, the
configuration files are copied to /etc/swift.
The only requirements are:
1. /etc/swift does not exist. That way the tests will
not interfere with an existing deployment.
2. /mnt/gluster-object/test and /mnt/gluster-object/test2
must have been created and setup correctly on an XFS
or GlusterFS volume
3. sudo rights without password prompt
4. glusterfs-openstack-swift-* rpm must not be installed
on the system
Once the requirements are met, you can execute the tests
as follows:
$ bash tools/functional_tests.sh
Change-Id: Icdbcd420355b02e64f294df7298a3e473b343655
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5281
Reviewed-by: Peter Portante <pportant@redhat.com>
Apparently, on RHEL 6, the valid archive type is tar, not tar.gz.
It causes git-archive and thus makerpm.sh to abort with:
fatal: Unknown archive format 'tar.gz'
To fix, use a regular pipeline.
Change-Id: I9174ad1d1c0e087f46ecd31eade61eeea75cb9cb
Signed-off-by: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Reviewed-on: http://review.gluster.org/5271
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This commit has the following changes:
* G4S no longer accepts URLs that end in /. A HTTP code
of 400 is returned when a / at the end of the object
is detected.
* Directories can be created as objects setting the
content-type to application/directory and content-length
to 0.
* Functional tests have been adjusted to work with
G4S constraints
Change-Id: I31038a59699a8e3eeaba902db322218c6400093e
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5246
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Community documentation is being written in Markdown
format because we can leverage GitHub.com's ability
to render Markdown into HTML directly from files
in the repo. Also, the GlusterFS Community project
has decided to use Markdown as an input into the
tool called pandoc which can convert the documents
into multiple formats.
Change-Id: Iec530f05a9a1ab3a95a1e97b791e8390068b99b4
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5256
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
It was painfully slow to gather a list of all the files in a directory tree
when there were as many directories as files, since we also stat'd all of the
directories as well. We only did that to cache using memcache the result, to
know when it changed. However, there was no way for a memcache object to be
passed down to this code. So we have removed the memcache support entirely,
and removed the stat of the directories as a result.
See BZ 911448 (https://bugzilla.redhat.com/show_bug.cgi?id=911448).
Change-Id: I2625f82eca36c31eeffa84dc76ef7f3b48aafec5
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/5252
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
This is a step towards making fd based system calls where ever possible to avoid
path lookups.
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Change-Id: I482ea29ebe0859d0a5307ff25ecb5945d54bc7ca
Reviewed-on: http://review.gluster.org/5251
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Copy the functional tests to our tree so that
we can edit and skip any tests we know we are
not going to support for this release
Change-Id: I93a76550aaaa58de49ec9a7178a34e081b7b7cf0
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5211
The code was raising an exception when the container (which
happens to be a directory) did not exist. To be compatible
with OpenStack Swift, we need to handle an object which its
container/directory does not exist.
BUG: 960944 (https://bugzilla.redhat.com/show_bug.cgi?id=960944)
Change-Id: Ibb2db354a655e040fb70ebbe6a7d8f815d33dc0f
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5201
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
