63 Commits

Author SHA1 Message Date
Prashanth Pai
eb7883dc23 Remove some dead code
Used vulture utility to detect unused code in swiftonfile.
https://pypi.python.org/pypi/vulture

Change-Id: I045f5a96a48d2384718ad0f993540caa3a866309
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-01-19 13:03:51 +05:30
Prashanth Pai
b4ee36ef11 Fix bug in object name constraint
SOF can support object names of upto 1024 if it conforms to the
following constraints:

Object names can have forward slashes ('/') in them. Each segment in
between these slashes cannot exceed 255 characters with exception of
the last segment which cannot exceed 221 characters.

This constraint arises from the fact that each segment except the last
one in object name is a directory, the last segment being an actual
file on the filesystem.

Also, restored default constraint values in swift.conf since the SOF
constraints middleware (always in proxy pipeline) will take care of it.

Change-Id: Ia7dc44671a87911c092fecd0344eace92f5c225b
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-12-01 13:05:29 +05:30
Thiago da Silva
c0b3d73189 new constraints middleware
Adding a constraints middleware that allows different constraints for a
Swift-on-File storage policy.

Currently, Swift allows only one set of object naming and metadata rules
per cluster. As new storage policies are implemented, it is possible that
different storage back-ends have different set of rules on how objects should
be created for that specific storage system. Swift-on-File has different rules
as objects name are mapped to a POSIX filesystem directory structure.
For example: no names with double slashes or ending with slashes are allowed.

At first a solution was proposed to include a generic patch in the upstream
swift code, but after discussing with the Swift community it became clear that
it would be better to have a global set of constraints that covers the whole
cluster and if a specific storage policy has specific constraints than it
should have its own middleware.
Link to patch for reference: https://review.openstack.org/#/c/113325/

Change-Id: I323ead5d98bf5c087930ccf446d3e8d83075e584
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-10-07 16:53:44 -04:00
Bill Owen
d187234c11 Rename exceptions to use SwiftOnFile name.
Rename exceptions from GlusterFS to SwiftOnFile names.

Change-Id: Idddc16777cd08e66ce6634a65e4199db1f67e20a
2014-09-29 22:32:47 -07:00
Thiago da Silva
16b7d16803 removing unused files
These files are no long in use by swiftonfile.
- functional_auth/tempauth files are no longer required because
tests now rely on a saio setup where tempauth is already used
by default.
- tools/functional_tests.sh has been replaced by .functests.
- tools/tox_run.sh is no longer required, but swiftonfile now requires
a saio setup and swiftonfile to be configured as a storage policy
for testing.
- test/unit/common/data/* files are also no longer in use.

Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-07-09 08:54:56 -04:00
Thiago da Silva
de24352275 changing references of gluster to swiftonfile
Renamed files and directories as well as replaced
infile references to gluster where appropriate

Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-07-08 11:04:31 -04:00
Prashanth Pai
abf0641e40 Support Storage Policy - Rev 3
* Functional tests now run in SAIO+SoF setup.
* Fix and update various constraints.
* Update object-server.conf, swift.conf files. Add test.conf file.
* Added ./.functests script

TODO:
* Reintroduce gluster_swift_tests.py with SoF
  specific functional tests.

Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-06-23 12:22:43 +05:30
Prashanth Pai
33e8e2312e Support Storage Policy - Rev 2
* Fix API inconsistencies
* Remove hard-coded reseller_prefix = AUTH_
* Remove unused methods from gluster/swift/common/utils.py
  These used to be called from DiskDir earlier.
* Remove hidden configuration option that were related to account
  and container listings and metadata.
* Remove unused and redundant attributes

Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-06-23 12:22:42 +05:30
Prashanth Pai
7ab8e35ff6 Support Storage Policy - Rev 1
* Inherit DiskFileMananger class and few other methods from Swift
  and reuse them.
* Unit tests for write_pickle method.
* Added policy_idx args, wherever missing.

Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-06-23 12:22:42 +05:30
Prashanth Pai
ba39ade716 Remove non SP related code
This commit only removes code that is not required when swiftonfile
will be used as Storage Policy in Swift.

This commit does NOT:
* include code changes to support SP
* fix functional tests
* update documentation related to SP

This commit removes the following:
* Gen builders script and its man page.
* Authentication guide, user guide and other docs.
* Swiftkerbauth code, unit tests, func tests and doc.
* gswauth code, unit tests, func tests and doc.
* Object expirer code modifications, unit tests and doc.
* Conf files of account, container, proxy.
* Account, container and proxy server code - server.py of each.
* Account, container and proxy unit and functional tests.
* DiskDir class and corresponding unit tests.
* Our overridden ring.py (enforces account = volume = device) and tests.
* Functional tests for authentication filters.
* modules/swift and test/deploy directories.
* Proxy base controllers unit tests.

NOTE: We may have to reintroduce some of the above functional and unit
tests after SP related code changes - during fixing functional tests.

This commit modifies:
* setup.py to reflect the above code removals.
* constraints.py to remove references to ring.py
* object server.py to remove object-expirer changes
* tox.ini to remove ksfunctest and swfunctest

Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-06-23 12:22:42 +05:30
Prashanth Pai
d23fd1b56c Sync with OpenStack Swift v1.13.1
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2014-05-08 11:45:08 +05:30
Thiago da Silva
404fcd815c Updating code to call fs_utils instead of python's os module
This change is being done to prepare the code to always call
fs_utils for all filesystem calls on the data being stored.
By creating this interface (i.e., fs_utils), we can then
make a seamless change to use the current method of 'os.'
calls over FUSE _or_ libgfapi. This new method will be introduced
in a new separate patch.

Change-Id: Ic768fa7352d7672b1f060230bb4486f0ec228152
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7333
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-04-10 12:50:38 -07:00
Thiago da Silva
b00a996732 removing exception handling in do_mkdir
This change is being made to allow callers to handle
any exception thrown by os.mkdir.

This function is currently never called anywhere in the code.
It was introduced as part of the first commit to this project
but it was never used.

This patch (http://review.gluster.org/#/c/5304/) removed the
early version of this function, and this patch
(http://review.gluster.org/#/c/5305/) added it back with new
exception handling.

Change-Id: I71325660cb47594b0804da3da21920e26d2055f2
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7350
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
Tested-by: Chetan Risbud <crisbud@redhat.com>
2014-03-27 03:14:32 -07:00
Thiago da Silva
972c252c41 removing check_user_xattr function
This function is no longer used, the code that called
this function was removed as part of this patch:
http://review.gluster.com/#/c/3390/

Change-Id: I6e81e4b763ce302289e2be5467dc4776f0750c15
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7336
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-03-26 08:25:40 -07:00
Prashanth Pai
2014cdb906 Add support for Object Expiration feature
Preventing access to expired objects
------------------------------------
Re-enabled accepting X-Delete-At and X-Delete-After headers. During a GET on
an expired object, DiskFileExpired is raised by DiskFile class. This will
result in object-server returning HTTPNotFound (404) to the client.

Tracking objects to be deleted
------------------------------
Objects to be deleted are tracked using "tracker objects". These are PUT into
a special account(a volume, for now). These zero size "tracker objects" have
names that contain:
    * Expiration timestamp
    * Path of the actual object to be deleted

Deleting actual objects from GlusterFS volume
---------------------------------------------
The object-expirer daemon runs a pass once every X seconds. For every pass it
makes, it queries the special account for "tracker objects". Based on
(timestamp, path) present in name of "tracker objects", object-expirer then
deletes the actual object and the corresponding tracker object.

To run object-expirer forever:
    swift-init object-expirer start
To run just once:
    swift-object-expirer -o -v /etc/swift/object-expirer.conf

Caveat/Limitation: Object-expirer needs a separate account(volume) that
is not used by other services like gswauth. By default, this volume is
named "gsexpiring" and is configurable.

More info about object expiration:
http://docs.openstack.org/developer/swift/overview_expiring_objects.html

Change-Id: I876995bf4f16ef4bfdff901561e0558ecf1dc38f
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6891
Tested-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
2014-03-24 22:14:15 -07:00
Chetan Risbud
03128e172e Rebase swiftkerbauth imported code with upstream
Few changes have been merged to upstream swiftkerbauth repo.
This commit brings it down to gluster-swift repo.
Bringing below changes to gluster-swift repo in one go.

http://review.gluster.org/#/c/6296/
http://review.gluster.org/#/c/6370/
http://review.gluster.org/#/c/6595/
http://review.gluster.org/#/c/6713/
http://review.gluster.org/#/c/6732/

Change-Id: I10dc12d75ec63fca313339fbc71e4f18071af552
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/6764
Reviewed-by: Prashanth Pai <ppai@redhat.com>
2014-01-27 22:55:55 -08:00
Chetan Risbud
4b988ce3c5 Initial import of the swiftkerbauth
Imported code till commit f64a3354185f32928e2568d9ece4a52fa4746c05
Changed a code bit to import correct definitions.
kerbauth unit tests do run along with gluster-swift.
Install script does install swiftkerbauth.
import swiftkerbauth from http://review.gluster.org/swiftkrbauth.git

Change-Id: Ia89f2b77cc68df10dee2f41ce074f3381ac3c408
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/6597
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-01-21 10:09:44 -08:00
Thiago da Silva
205a6e4aa7 Fix bug where admin is able to delete reseller_admin
Changed the code to only allow the super_admin to delete a reseller_admin.
This follows the same logic of user creation, where only the super_admin
can create a reseller_admin.

Also, took the opportunity to refactor some code and implemented
get_user_detail method to remove duplicated code

Bug 1260239: https://bugs.launchpad.net/gluster-swift/+bug/1260239

Change-Id: I9e4866cd7ad08698f427846be566ab2364ad4850
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6516
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-01-10 12:52:53 -08:00
Prashanth Pai
94a3f539e7 Fix users not able to change their own password/key
Users were not able to update their own password/key
with the update operation resulting in 403 (HTTPForbidden).

EXAMPLES:
Command to update password/key of regular user:
gswauth-add-user -U account1:user1 -K old_pass account1 user1 new_pass

Command to update password/key of account admin:
gswauth-add-user -U account1:admin -K old_pass -a account1 admin new_pass

Command to update password/key of reseller_admin:
gswauth-add-user -U account1:radmin -K old_pass -r account1 radmin new_pass

BUG: https://bugs.launchpad.net/gluster-swift/+bug/1262227

Change-Id: I604da5aee67099b29541eb7e51a040a041f1961b
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6650
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-01-07 13:50:30 -08:00
Prashanth Pai
d1c7b1cc4b Handle ENOSPC on os.close() operation
It is quite possible that errors on a previous write()
operation are first reported at the final close().

For fruther info, refer to: http://review.gluster.org/#/c/6269/

Change-Id: If0fbe2f5109d28c82cb493f2526fd5057f86b556
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6608
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2014-01-07 13:49:29 -08:00
Prashanth Pai
a8c84cb2da gswauth: Fix 403 being returned instead of 401
- 401(Unauthorized) is to be returned when user credentials are
  wrong where as 403(Forbidden) is to be returned when user
  credentials are correct but the user doesn't have the priveleges
  to carry out the operation.

- Also error messages displayed when using swauth-* command line
  utilities have been updated.

Change-Id: I485786896ad14d3263f4325d1857cacc93adab96
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6336
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6447
Reviewed-by: Thiago Da Silva <thiago@redhat.com>
Tested-by: Thiago Da Silva <thiago@redhat.com>
2013-12-18 04:12:43 -08:00
Luis Pabon
5562d18fed CentOS python-2.6 fixes
* Need to replace functions which are not supported in python 2.6

Change-Id: I4308f7c43a60fde9d8538b53422ddc18499dc1f9
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6457
Reviewed-on: http://review.gluster.org/6484
2013-12-11 10:32:42 -08:00
Thiago da Silva
2d818c06b8 fix issue with swauth-clean-token returning 403 errors
The issue was due to missing a necessary change that was made when changing
the auth account name from .auth to metadata volume.

the auth account has a group of the same name, so the .auth account
also had a .auth group, so we needed to change that too to the
metadata volume (e.g., gsmetadata)

Change-Id: Iaa3b7a1b2628f5b863807932e863593be0011a82
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6416
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-12-05 09:18:30 -08:00
Luis Pabon
b36fe03702 Return BadRequest on X-Delete-At/After headers
Gluster-swift does not support X-Delete-After or X-Delete-At
headers.  The code needs to inform the caller with a 400-BadRequest
if they use these headers.

Change-Id: Ic9d3a1646c0d26bb0204245efce4501f7479fee6
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6364
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
2013-12-05 04:50:38 -08:00
Thiago da Silva
f8ae648c6d fix error with swauth-clean-tokens
Error is caused by left-over code from swauth. It had hard-coded
the auth account to .auth. Changed code to use volume specified
in the config file

Change-Id: Id00eb3964b6b26a84511e4753ab80b2dea5654da
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6280
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-11-19 19:10:36 -08:00
Prashanth Pai
c6d7ddc4bc Improve logging and raising DiskFileNoSpace
This commit only improves logging whenever ENOSPC (No space on disk)
or EDQUOT (Quota limit exceeded) is returned by glusterfs

Also, added methods to:
- get filename from file descriptor
- log with rate limit

Caveat: Although raising DiskFileNoSpace results in object-server
returning HTTPInsufficientStorage[507] correctly, the swift proxy-server
invokes "best_response" method that returns [503] to the user.

When write-behind translator is turned on in glusterfs, it may set
errno to EIO instead of ENOSPC/EDQUOT. This is documented in BZ 986812

BUG: 985862, 985253, 1020724
Change-Id: Ib0c5e41c11a8cdccc2077f71c31d8a23229452bb
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6199
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6269
2013-11-15 09:06:21 -08:00
Thiago da Silva
a5c772955f make gluster volume name configurable
changed code to look at configuration file for
the metadata gluster volume name. The default is still
gsmetadata

Change-Id: I9bf74b9566ea1c9716c42f7ced0f999e02824868
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6225
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-11-05 17:23:33 -08:00
Thiago da Silva
9f8d2e61a7 first gswauth functional tests
commiting first gswauth functional tests.
Currently there are two tests, to create account
and to create an user. Each test is self contained
in that it goes through the process of creating and deleting
accounts and users as needed.

More tests will be added shortly.

Change-Id: I26d577790aed8c79c9de11f224516423e9769962
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6188
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-30 16:57:46 -07:00
Peter Portante
100d6b01bd Rebase to lastest OpenStack Swift DiskFile API
As of October 28th, 2013, we rebase to OpenStack Swift master (commit
4bfe674) to pick up the lastest officially supported DiskFile API
changes. We use a snapshot of OpenStack Swift stored in the
gluster-swift launchpad downloads area so that we can deliberately
rebase at our own pace.

With this refactoring, all the code for handling I/O is wholly
contained in the swift tree for object operations. This will allow us
to use a different fs_utils implementation in the future (for example,
one based on a yet-to-be-implemented python bindings over libgfapi).
This also means the "Fake_file" class has been removed entirely.

Change-Id: I767983f88c59786e30b6c64da16d1cb6ab3c3e7f
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5993
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-29 07:25:59 -07:00
Peter Portante
286a1308db Rebase to OpenStack Swift Havana (1.10.0)
Change-Id: I90821230a1a7100c74d97cccc9c445251d0f65e7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/6157
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-28 11:51:51 -07:00
Thiago da Silva
b48149a4af applied changes made by Luis Pabon to support gluster-swift
The current changes support only one account per volume
The next changes will be made to add support for multiple accounts
Also modified unit tests that were failing after code changes.
Original changes were made here: https://github.com/lpabon/swauth/tree/gluster-swift

Change-Id: I5577d7dc042fc22de8625c8bdf30329cc3338cb9
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6107
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-24 11:49:23 -07:00
Thiago da Silva
cadaed4627 moving existing swauth unit tests
moving existing swauth unit tests
to gluster-swift unit tests location

Change-Id: I3445b7ef1a1abe584854f2b04ffc9949b3346814
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6106
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-17 12:59:48 -07:00
Thiago da Silva
46cd43fdf4 Adding new unit tests and removed some unused functions
New unit tests help improve test coverage of common/Glusterfs.py code
Also removed a function that was no longer being used.

Change-Id: Iaa0eed3d2b9ffcc148c1e00b28322ebf93b3f13c
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6053
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-10-10 13:42:31 -07:00
Peter Portante
b00f10f5ca Fix container details test's starting environment
When the unit tests are run in an environment like the functional
tests, where the accurrate size in listings switch is on, we need to
sure that the tests that verify the behavior when it is off run in
that state.

Change-Id: I9f3ef90af7ef4edc698e2b8b7f016e6aaa7fac98
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5703
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-08-28 19:18:36 -07:00
Luis Pabon
3861bcec94 Depend only on OpenStack Swift 1.9.1
* The master branch is currently tracking the development
version 1.9.1 of Havana OpenStack Swift.

* Seems that we still have some settings which are not
reset in the unit tests.  This fix provides a guaranteed
state for test_utils.py:TestUtils.test_get_container_details

Change-Id: Ie32f49743301bfa493f70ed8a539fb6840f9f67b
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5719
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-08-28 19:01:50 -07:00
Peter Portante
9d4e67e741 Updates to support Havana interim version 1.9.1.
The code changes are basically:

  * Apply refactoring in the DiskFile class to use the new DiskWriter
    abstraction

    * Move and rename our diskfile module to match upstream

  * ThreadPools allow us to remove the tpool usage around fsync

  * Update the Ring subclass to support the get_part() method

  * Update to use the 1.9.1 proxy server unit tests

    * Move the DebugLogger class to test.unit

  * Rebuild the Rings to use the new layout

    * Remove backup ring builder files

  * Update spec files to 1.9.1, and tox to use swift 1.9.1

  * Updated version to 1.9.0-0

Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5331
2013-08-21 19:38:35 -07:00
Luis Pabon
9e6fbbcee9 perf: Container and account performance inc
* Container and accounts performance increase by removing
  the need to update either the object count or the
  container count, respectively.  New hidden
  configuratoins added to re-enable the functionality.
* object_only configuratoin removed from fs.conf and
  replaced with a hidden configuration.  The new hidden
  configuration reports gratuituosly crated directories
  as objects, to support a compatibility behavior with
  previous version 1.4.8 (even though it was incorrect).

BUG 988969: https://bugzilla.redhat.com/show_bug.cgi?id=988969

Change-Id: Idca20b8629ec38606ff5692fe62bc2cadabffc86
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5403
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-on: http://review.gluster.org/5504
2013-08-06 11:23:46 -07:00
Chetan Risbud
027951c102 Gluster to handle ENOSPC (Error 28) correctly
A gluster volume could yield an ENOSPC condition seeing
that a volume is full.  This needed to handled correctly.
Added error handling.

BUG: 985253

https://bugzilla.redhat.com/show_bug.cgi?id=985253

Change-Id: I85472c0a81a354a2796327fead606da3a938d4bf
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5362
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-08-06 03:50:59 -07:00
Chetan Risbud
a71ab7b9b7 Updating copyrights with current year
Copyrights are updated across the code to reflect 2013.
Copyrights message was added to couple of scripts.

Change-Id: I2cfc7bdc6adaa5c9ab0075aab49021a5f0dea7a1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5371
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-07-24 18:43:52 -07:00
Mohammed Junaid
b1919bd794 Gracefully handle the ring file not found error.
Change-Id: I35c33f3938709d5d9a9a55640a503f186df9a965
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/5368
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-07-24 18:41:54 -07:00
Peter Portante
677d307169 Final forward port of PDQ performance patches
Change-Id: I4ef131b3cc7648d4571a4d854029efb1aff8b901
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5305
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-07-10 15:49:26 -07:00
Luis Pabon
3f72ae45a3 Remove do_mkdir and simplify mkdirs
Change-Id: I2e629533bceafb341ced5b1b3f2436448293e03f
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5304
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-07-09 18:20:05 -07:00
Luis Pabon
a737372032 Removed unused metadata key and do_rmdir func
Change-Id: Ibcef5d817390a11d85cf125bb9addfbfded1b019
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5301
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-07-09 04:11:29 -07:00
Luis Pabon
5cef798f8d OpenStack Swift Functional Tests for G4S
This commit has the following changes:
  * G4S no longer accepts URLs that end in /.  A HTTP code
of 400 is returned when a / at the end of the object
is detected.

  * Directories can be created as objects setting the
content-type to application/directory and content-length
to 0.

  * Functional tests have been adjusted to work with
G4S constraints

Change-Id: I31038a59699a8e3eeaba902db322218c6400093e
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5246
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-06-28 13:17:50 -07:00
Mohammed Junaid
cc97abca04 object-storage: remove stat of directories
It was painfully slow to gather a list of all the files in a directory tree
when there were as many directories as files, since we also stat'd all of the
directories as well. We only did that to cache using memcache the result, to
know when it changed. However, there was no way for a memcache object to be
passed down to this code. So we have removed the memcache support entirely,
and removed the stat of the directories as a result.

See BZ 911448 (https://bugzilla.redhat.com/show_bug.cgi?id=911448).

Change-Id: I2625f82eca36c31eeffa84dc76ef7f3b48aafec5
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/5252
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-06-27 17:21:18 -07:00
Mohammed Junaid
c7ce3beec1 object-storage: Use fchown instead of chown.
This is a step towards making fd based system calls where ever possible to avoid
path lookups.

Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Change-Id: I482ea29ebe0859d0a5307ff25ecb5945d54bc7ca
Reviewed-on: http://review.gluster.org/5251
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-06-27 17:15:45 -07:00
Luis Pabon
b00e479637 Return correct status when deleting non-existing container
The code was raising an exception when the container (which
happens to be a directory) did not exist.  To be compatible
with OpenStack Swift, we need to handle an object which its
container/directory does not exist.

BUG: 960944 (https://bugzilla.redhat.com/show_bug.cgi?id=960944)

Change-Id: Ibb2db354a655e040fb70ebbe6a7d8f815d33dc0f
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5201
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-06-11 13:39:53 -07:00
Peter Portante
a88accbe27 Refactor DiskDir and DiskAccount to use DiskCommon
We have refactored DiskDir and DiskAccount so that truly common,
shared functions live in the base class DiskCommon, and each of these
then sub-class that. This allows us to isolate unique behaviors.

We have also enabled all the skipped unit tests in test_diskdir.py
that we could, removing a few that we don't need right now.

Change-Id: I48d9c915108df8cc92c3e9a764563e1d10c50050
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5148
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
2013-06-07 04:29:35 -07:00
Peter Portante
f80872e852 Change filters to use a generator pattern
By using a generator pattern, we avoid creating whole new lists each
time, instead we iterate through the original list once (after it is
sorted), constructing the final list only once.

We also address the behavioral differences between the swift filtering
results and our code so that ported unit tests work the same
(non-slash objects, that is).

Change-Id: If32c1987f24781ff81ab4c28c9ddfff17c2e7787
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5145
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
2013-06-06 12:37:45 -07:00
Luis Pabon
40c313378e Copy proxy unit test from OpenStack Swift
This is a copy of the proxy unit test from OpenStack
Swift.  The file has minor changes to make it compatible
with Gluster For Swift.

Change-Id: I64a0f8d274461eb2a2c38524c6282e0d3d3d1457
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5113
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-06-03 16:52:15 -07:00