The file descriptor is not being closed because
it is self._fd is None
Change-Id: I7edc8a78b09bdd76d59ac8f3dbc809af652f9b0e
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7315
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This function is no longer used, the code that called
this function was removed as part of this patch:
http://review.gluster.com/#/c/3390/
Change-Id: I6e81e4b763ce302289e2be5467dc4776f0750c15
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7336
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Preventing access to expired objects
------------------------------------
Re-enabled accepting X-Delete-At and X-Delete-After headers. During a GET on
an expired object, DiskFileExpired is raised by DiskFile class. This will
result in object-server returning HTTPNotFound (404) to the client.
Tracking objects to be deleted
------------------------------
Objects to be deleted are tracked using "tracker objects". These are PUT into
a special account(a volume, for now). These zero size "tracker objects" have
names that contain:
* Expiration timestamp
* Path of the actual object to be deleted
Deleting actual objects from GlusterFS volume
---------------------------------------------
The object-expirer daemon runs a pass once every X seconds. For every pass it
makes, it queries the special account for "tracker objects". Based on
(timestamp, path) present in name of "tracker objects", object-expirer then
deletes the actual object and the corresponding tracker object.
To run object-expirer forever:
swift-init object-expirer start
To run just once:
swift-object-expirer -o -v /etc/swift/object-expirer.conf
Caveat/Limitation: Object-expirer needs a separate account(volume) that
is not used by other services like gswauth. By default, this volume is
named "gsexpiring" and is configurable.
More info about object expiration:
http://docs.openstack.org/developer/swift/overview_expiring_objects.html
Change-Id: I876995bf4f16ef4bfdff901561e0558ecf1dc38f
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6891
Tested-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Also, bumped version of gluster-swift to v1.13.0
Change-Id: I797dc704c9523540cba847b1e8ff3da97b79630c
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/7229
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
PUT of a directory fails with gluster-swift when there exists
object of a same name. This is because the objects and directories are placed on
the glusterfs. And hence the filesystem semantics are applicable there.
Exceptions raised in such situation are needed to be handled and reported
correctly back to proxy-server with HTTPConflict as a error code. Although
swift still continues to choose 503 as the best respose in such cases. No
tracebacks reported.
Fix covers the case when there exists a directory and object of the same
name is PUT.
Code changes fixes both the failure cases mentioned in the bug.
Examples of failing PUT requests:
1) curl -v -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1/dir1/obj2/anotherobject
-d'asdasdsadA'
-- obj2 was already an object.
2)curl -v -H 'Content-Length: 0' -H 'Content-Type: application/directory' -X PUT
http://127.0.0.1:8080/v1/AUTH_test/c1/obj1
-- obj1 was already and object
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1071021
Change-Id: Id3042d920e3f99e740d4042ef5907ac8c59e04db
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/7181
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Prashanth Pai <ppai@redhat.com>
* Bumped the version of gluster-swift to v1.12.0.
* Added document on how to do a sync
Change-Id: I676e110759b0bce3007a2a38f3b384b4ca9d710d
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6977
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Tested-by: Chetan Risbud <crisbud@redhat.com>
Imported code till commit f64a3354185f32928e2568d9ece4a52fa4746c05
Changed a code bit to import correct definitions.
kerbauth unit tests do run along with gluster-swift.
Install script does install swiftkerbauth.
import swiftkerbauth from http://review.gluster.org/swiftkrbauth.git
Change-Id: Ia89f2b77cc68df10dee2f41ce074f3381ac3c408
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/6597
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Changed the code to only allow the super_admin to delete a reseller_admin.
This follows the same logic of user creation, where only the super_admin
can create a reseller_admin.
Also, took the opportunity to refactor some code and implemented
get_user_detail method to remove duplicated code
Bug 1260239: https://bugs.launchpad.net/gluster-swift/+bug/1260239
Change-Id: I9e4866cd7ad08698f427846be566ab2364ad4850
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6516
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Users were not able to update their own password/key
with the update operation resulting in 403 (HTTPForbidden).
EXAMPLES:
Command to update password/key of regular user:
gswauth-add-user -U account1:user1 -K old_pass account1 user1 new_pass
Command to update password/key of account admin:
gswauth-add-user -U account1:admin -K old_pass -a account1 admin new_pass
Command to update password/key of reseller_admin:
gswauth-add-user -U account1:radmin -K old_pass -r account1 radmin new_pass
BUG: https://bugs.launchpad.net/gluster-swift/+bug/1262227
Change-Id: I604da5aee67099b29541eb7e51a040a041f1961b
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6650
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
It is quite possible that errors on a previous write()
operation are first reported at the final close().
For fruther info, refer to: http://review.gluster.org/#/c/6269/
Change-Id: If0fbe2f5109d28c82cb493f2526fd5057f86b556
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6608
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
- 401(Unauthorized) is to be returned when user credentials are
wrong where as 403(Forbidden) is to be returned when user
credentials are correct but the user doesn't have the priveleges
to carry out the operation.
- Also error messages displayed when using swauth-* command line
utilities have been updated.
Change-Id: I485786896ad14d3263f4325d1857cacc93adab96
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6336
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6447
Reviewed-by: Thiago Da Silva <thiago@redhat.com>
Tested-by: Thiago Da Silva <thiago@redhat.com>
The DiskWriter was closing the file descriptor when it finished
writing but initializing it to None. Therefore, the DiskFile
context manager would then try to close it also.
Change-Id: I188ec814d025e28c7b89532f0502ebf1d4a20a09
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6317
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6487
* Need to replace functions which are not supported in python 2.6
Change-Id: I4308f7c43a60fde9d8538b53422ddc18499dc1f9
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6457
Reviewed-on: http://review.gluster.org/6484
* Updated Proxy unit test
* Updated Functional tests
* Updated Tox to point to the new swift snapshot available
on http://launchpad.net/gluster-swift
Change-Id: Ia91593c6a28d5a3fe70715ddc60546931ae71635
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6445
Unit tests failed as upstream swift was updated with this commit:
f0c0855ec8207fa8be63ed4f3135c8e29a31acc0
Updated gluster-swift unit tests accordingly.
Change-Id: I286c807a17d79c2d9a21ee436f029a0f59cb9b7b
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6268
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The issue was due to missing a necessary change that was made when changing
the auth account name from .auth to metadata volume.
the auth account has a group of the same name, so the .auth account
also had a .auth group, so we needed to change that too to the
metadata volume (e.g., gsmetadata)
Change-Id: Iaa3b7a1b2628f5b863807932e863593be0011a82
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6416
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Gluster-swift does not support X-Delete-After or X-Delete-At
headers. The code needs to inform the caller with a 400-BadRequest
if they use these headers.
Change-Id: Ic9d3a1646c0d26bb0204245efce4501f7479fee6
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6364
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
During a post request, the object-server is ovewriting
the existing object metadata. This fix prevents the overwrite
of the system metadata while it allows for the user to
add/remove user metadata
Change-Id: Ic62cd064589b625ee425a9934be8766650622c13
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6254
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6315
Error is caused by left-over code from swauth. It had hard-coded
the auth account to .auth. Changed code to use volume specified
in the config file
Change-Id: Id00eb3964b6b26a84511e4753ab80b2dea5654da
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6280
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This commit only improves logging whenever ENOSPC (No space on disk)
or EDQUOT (Quota limit exceeded) is returned by glusterfs
Also, added methods to:
- get filename from file descriptor
- log with rate limit
Caveat: Although raising DiskFileNoSpace results in object-server
returning HTTPInsufficientStorage[507] correctly, the swift proxy-server
invokes "best_response" method that returns [503] to the user.
When write-behind translator is turned on in glusterfs, it may set
errno to EIO instead of ENOSPC/EDQUOT. This is documented in BZ 986812
BUG: 985862, 985253, 1020724
Change-Id: Ib0c5e41c11a8cdccc2077f71c31d8a23229452bb
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6199
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6269
changed code to look at configuration file for
the metadata gluster volume name. The default is still
gsmetadata
Change-Id: I9bf74b9566ea1c9716c42f7ced0f999e02824868
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6225
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
commiting first gswauth functional tests.
Currently there are two tests, to create account
and to create an user. Each test is self contained
in that it goes through the process of creating and deleting
accounts and users as needed.
More tests will be added shortly.
Change-Id: I26d577790aed8c79c9de11f224516423e9769962
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6188
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
As of October 28th, 2013, we rebase to OpenStack Swift master (commit
4bfe674) to pick up the lastest officially supported DiskFile API
changes. We use a snapshot of OpenStack Swift stored in the
gluster-swift launchpad downloads area so that we can deliberately
rebase at our own pace.
With this refactoring, all the code for handling I/O is wholly
contained in the swift tree for object operations. This will allow us
to use a different fs_utils implementation in the future (for example,
one based on a yet-to-be-implemented python bindings over libgfapi).
This also means the "Fake_file" class has been removed entirely.
Change-Id: I767983f88c59786e30b6c64da16d1cb6ab3c3e7f
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5993
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The current changes support only one account per volume
The next changes will be made to add support for multiple accounts
Also modified unit tests that were failing after code changes.
Original changes were made here: https://github.com/lpabon/swauth/tree/gluster-swift
Change-Id: I5577d7dc042fc22de8625c8bdf30329cc3338cb9
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6107
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The modified test case provides the complete coverage over
Fake_file class. These interfaces are to be used for directory
type of the objects.
BUG:987841
https://bugzilla.redhat.com/show_bug.cgi?id=987841
Change-Id: I2401423d2013ce7d90c454c72fbb60f4bc05ceb1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/6037
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
moving existing swauth unit tests
to gluster-swift unit tests location
Change-Id: I3445b7ef1a1abe584854f2b04ffc9949b3346814
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6106
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
New unit tests help improve test coverage of common/Glusterfs.py code
Also removed a function that was no longer being used.
Change-Id: Iaa0eed3d2b9ffcc148c1e00b28322ebf93b3f13c
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6053
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The version manifest unit test needed to be adjusted
for gluster-swift. Gluster-swift does not support a PUT
of an object "a" then another object "a/b". The first object
"a" would be interpreted as a file, while "a/b" would
require "a" to now be a directory.
Change-Id: I1628d5575d1b68ccce20386a4a0a0b2442d631af
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5987
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
These are a copy of the OpenStack Swift proxy controller
unit tests as of version their 1.9.1.
Change-Id: Ib05bc2a37dbb29d729346d78ea8f6de83b82375b
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5735
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Added a fake_file class that implements minimal set of functions that
are invoked by the code in GET.
BUG: 987841
https://bugzilla.redhat.com/show_bug.cgi?id=987841
Change-Id: I5bdf5be1c0c4c8231f34c9be529e6edc83774f2e
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5511
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
For whatever reason, it appears that GlusterFS, or perhaps FUSE can
continuously return ENOENT on a rename system call even when we have
double checked that there is no reason to do so. That is a bug for
that sub system. However, our response to that bug can result in an
infinite loop, which is bad.
This code reduces that to 10 attempts.
In addition, we restructed the open retry loop to match, providing
module constants for the upper bounds of both retry loops.
BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX)
Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5670
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
When the unit tests are run in an environment like the functional
tests, where the accurrate size in listings switch is on, we need to
sure that the tests that verify the behavior when it is off run in
that state.
Change-Id: I9f3ef90af7ef4edc698e2b8b7f016e6aaa7fac98
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5703
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
* The master branch is currently tracking the development
version 1.9.1 of Havana OpenStack Swift.
* Seems that we still have some settings which are not
reset in the unit tests. This fix provides a guaranteed
state for test_utils.py:TestUtils.test_get_container_details
Change-Id: Ie32f49743301bfa493f70ed8a539fb6840f9f67b
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5719
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
The code changes are basically:
* Apply refactoring in the DiskFile class to use the new DiskWriter
abstraction
* Move and rename our diskfile module to match upstream
* ThreadPools allow us to remove the tpool usage around fsync
* Update the Ring subclass to support the get_part() method
* Update to use the 1.9.1 proxy server unit tests
* Move the DebugLogger class to test.unit
* Rebuild the Rings to use the new layout
* Remove backup ring builder files
* Update spec files to 1.9.1, and tox to use swift 1.9.1
* Updated version to 1.9.0-0
Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5331
Our initial implementation only required Jenkins
to export binary RPMs, but as we move foward, we really
need to also export SRPMs. To support SRPMs, the spec
file in the RPM has to have the correct NAME, VERSION,
and RELEASE information.
Change-Id: Icd7132b4aafdbe7a1f02a35d0be7ad63b2e7c056
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5669
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-on: http://review.gluster.org/5679
* Container and accounts performance increase by removing
the need to update either the object count or the
container count, respectively. New hidden
configuratoins added to re-enable the functionality.
* object_only configuratoin removed from fs.conf and
replaced with a hidden configuration. The new hidden
configuration reports gratuituosly crated directories
as objects, to support a compatibility behavior with
previous version 1.4.8 (even though it was incorrect).
BUG 988969: https://bugzilla.redhat.com/show_bug.cgi?id=988969
Change-Id: Idca20b8629ec38606ff5692fe62bc2cadabffc86
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5403
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-on: http://review.gluster.org/5504
A gluster volume could yield an ENOSPC condition seeing
that a volume is full. This needed to handled correctly.
Added error handling.
BUG: 985253
https://bugzilla.redhat.com/show_bug.cgi?id=985253
Change-Id: I85472c0a81a354a2796327fead606da3a938d4bf
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5362
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Copyrights are updated across the code to reflect 2013.
Copyrights message was added to couple of scripts.
Change-Id: I2cfc7bdc6adaa5c9ab0075aab49021a5f0dea7a1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5371
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Change-Id: Ibcef5d817390a11d85cf125bb9addfbfded1b019
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5301
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
This commit has the following changes:
* G4S no longer accepts URLs that end in /. A HTTP code
of 400 is returned when a / at the end of the object
is detected.
* Directories can be created as objects setting the
content-type to application/directory and content-length
to 0.
* Functional tests have been adjusted to work with
G4S constraints
Change-Id: I31038a59699a8e3eeaba902db322218c6400093e
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5246
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
