swift: Passing account name in container_update.

This patch also contains fixes to bugs
  * 811501
  * 812498
  * 821310

Also, removed the default set of users in the proxy-server.conf file.

Change-Id: Ief83905d10ff7bf7c43685ada4d7f05959cee9d1
BUG: 821310
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.com/3390
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
This commit is contained in:
Mohammed Junaid 2012-05-21 15:47:28 +05:30 committed by Peter Portante
parent 0eed5dd234
commit e625cc40ff
8 changed files with 1282 additions and 799 deletions

View File

@ -0,0 +1,60 @@
############################################################################################################
# Command to build rpms.#
# $ rpmbuild -ta %{name}-%{version}-%{release}.tar.gz #
############################################################################################################
# Setting up the environment. #
# * Create a directory %{name}-%{version} under $HOME/rpmbuild/SOURCES #
# * Copy the contents of plugins directory into $HOME/rpmbuild/SOURCES/%{name}-%{version} #
# * tar zcvf %{name}-%{version}-%{release}.tar.gz $HOME/rpmbuild/SOURCES/%{name}-%{version} %{name}.spec #
# For more information refer #
# http://fedoraproject.org/wiki/How_to_create_an_RPM_package #
############################################################################################################
%define _confdir /etc/swift
%define _swiftdir /usr/lib/python2.6/site-packages/swift
%define _ufo_version 1.0
%define _ufo_release 3
Summary : GlusterFS Unified File and Object Storage.
Name : gluster-swift-plugin
Version : %{_ufo_version}
Release : %{_ufo_release}
Group : Application/File
Vendor : Red Hat Inc.
Source0 : %{name}-%{version}-%{release}.tar.gz
Packager : gluster-users@gluster.org
License : Apache
BuildArch: noarch
Requires : memcached
Requires : openssl
Requires : python
Requires : gluster-swift
%description
Gluster Unified File and Object Storage unifies NAS and object storage
technology. This provides a system for data storage that enables users to access
the same data as an object and as a file, simplifying management and controlling
storage costs.
%prep
%setup -q
%install
rm -rf %{buildroot}
mkdir -p %{buildroot}/%{_swiftdir}/plugins
mkdir -p %{buildroot}/%{_confdir}/
cp constraints.py %{buildroot}/%{_swiftdir}/plugins
cp DiskDir.py %{buildroot}/%{_swiftdir}/plugins
cp DiskFile.py %{buildroot}/%{_swiftdir}/plugins
cp Glusterfs.py %{buildroot}/%{_swiftdir}/plugins
cp __init__.py %{buildroot}/%{_swiftdir}/plugins
cp utils.py %{buildroot}/%{_swiftdir}/plugins
cp -r conf/* %{buildroot}/%{_confdir}/
%files
%defattr(-,root,root)
%{_swiftdir}/plugins
%{_confdir}/

View File

@ -0,0 +1,396 @@
%if ! (0%{?fedora} > 12 || 0%{?rhel} > 5)
%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
%endif
Name: gluster-swift
Version: 1.4.8
Release: 3%{?dist}
Summary: OpenStack Object Storage (swift)
Group: Development/Languages
License: ASL 2.0
URL: http://launchpad.net/swift
Source0: http://launchpad.net/swift/essex/%{version}/+download/swift-%{version}.tar.gz
Source1: %{name}-functions
Source2: %{name}-account.init
Source4: %{name}-container.init
Source5: %{name}-object.init
Source6: %{name}-proxy.init
Patch0: openstack-swift-newdeps.patch
Patch1: openstack-swift-docmod.patch
Patch2: openstack-swift-nonet.patch
Patch3: gluster.patch
BuildRoot: %{_tmppath}/swift-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
BuildRequires: dos2unix
BuildRequires: python-devel
BuildRequires: python-setuptools
BuildRequires: python-netifaces
BuildRequires: python-paste-deploy
Requires: python-configobj
Requires: python-eventlet >= 0.9.8
Requires: python-greenlet >= 0.3.1
Requires: python-paste-deploy
Requires: python-simplejson
Requires: python-webob1.0
Requires: pyxattr
Requires: python-setuptools
Requires: python-netifaces
Requires: python-netifaces
Conflicts: openstack-swift
Requires(post): chkconfig
Requires(postun): initscripts
Requires(preun): chkconfig
Requires(pre): shadow-utils
Obsoletes: openstack-swift-auth <= 1.4.0
%description
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
Objects are written to multiple hardware devices in the data center, with the
OpenStack software responsible for ensuring data replication and integrity
across the cluster. Storage clusters can scale horizontally by adding new nodes,
which are automatically configured. Should a node fail, OpenStack works to
replicate its content from other active nodes. Because OpenStack uses software
logic to ensure data replication and distribution across different devices,
inexpensive commodity hard drives and servers can be used in lieu of more
expensive equipment.
%package account
Summary: A swift account server
Group: Applications/System
Requires: %{name} = %{version}-%{release}
%description account
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
This package contains the %{name} account server.
%package container
Summary: A swift container server
Group: Applications/System
Requires: %{name} = %{version}-%{release}
%description container
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
This package contains the %{name} container server.
%package object
Summary: A swift object server
Group: Applications/System
Requires: %{name} = %{version}-%{release}
Requires: rsync >= 3.0
%description object
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
This package contains the %{name} object server.
%package proxy
Summary: A swift proxy server
Group: Applications/System
Requires: %{name} = %{version}-%{release}
%description proxy
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
This package contains the %{name} proxy server.
%package doc
Summary: Documentation for %{name}
Group: Documentation
#%if 0%{?rhel} >= 6
#BuildRequires: python-sphinx10 >= 1.0
#%endif
%if 0%{?fedora} >= 14
BuildRequires: python-sphinx >= 1.0
%endif
# Required for generating docs
BuildRequires: python-eventlet
BuildRequires: python-simplejson
BuildRequires: python-webob1.0
BuildRequires: pyxattr
%description doc
OpenStack Object Storage (swift) aggregates commodity servers to work together
in clusters for reliable, redundant, and large-scale storage of static objects.
This package contains documentation files for %{name}.
%prep
%setup -q -n swift-%{version}
%patch0 -p1 -b .newdeps
%patch1 -p1 -b .docmod
%patch2 -p1 -b .nonet
%patch3 -p1 -b .gluster
# Fix wrong-file-end-of-line-encoding warning
dos2unix LICENSE
%build
%{__python} setup.py build
# Fails unless we create the build directory
mkdir -p doc/build
# Build docs
%if 0%{?fedora} >= 14
%{__python} setup.py build_sphinx
%endif
#%if 0%{?rhel} >= 6
#export PYTHONPATH="$( pwd ):$PYTHONPATH"
#SPHINX_DEBUG=1 sphinx-1.0-build -b html doc/source doc/build/html
#SPHINX_DEBUG=1 sphinx-1.0-build -b man doc/source doc/build/man
#%endif
# Fix hidden-file-or-dir warning
#rm doc/build/html/.buildinfo
%install
rm -rf %{buildroot}
%{__python} setup.py install -O1 --skip-build --root %{buildroot}
# Init helper functions
install -p -D -m 644 %{SOURCE1} %{buildroot}%{_datarootdir}/%{name}/functions
# Init scripts
install -p -D -m 755 %{SOURCE2} %{buildroot}%{_initrddir}/%{name}-account
install -p -D -m 755 %{SOURCE4} %{buildroot}%{_initrddir}/%{name}-container
install -p -D -m 755 %{SOURCE5} %{buildroot}%{_initrddir}/%{name}-object
install -p -D -m 755 %{SOURCE6} %{buildroot}%{_initrddir}/%{name}-proxy
# Remove tests
rm -fr %{buildroot}/%{python_sitelib}/test
# Misc other
install -d -m 755 %{buildroot}%{_sysconfdir}/swift
install -d -m 755 %{buildroot}%{_sysconfdir}/swift/account-server
install -d -m 755 %{buildroot}%{_sysconfdir}/swift/container-server
install -d -m 755 %{buildroot}%{_sysconfdir}/swift/object-server
install -d -m 755 %{buildroot}%{_sysconfdir}/swift/proxy-server
# Install pid directory
install -d -m 755 %{buildroot}%{_localstatedir}/run/swift
install -d -m 755 %{buildroot}%{_localstatedir}/run/swift/account-server
install -d -m 755 %{buildroot}%{_localstatedir}/run/swift/container-server
install -d -m 755 %{buildroot}%{_localstatedir}/run/swift/object-server
install -d -m 755 %{buildroot}%{_localstatedir}/run/swift/proxy-server
%clean
rm -rf %{buildroot}
%pre
getent group swift >/dev/null || groupadd -r swift -g 160
getent passwd swift >/dev/null || \
useradd -r -g swift -u 160 -d %{_sharedstatedir}/swift -s /sbin/nologin \
-c "OpenStack Swift Daemons" swift
exit 0
%post account
/sbin/chkconfig --add %{name}-account
%preun account
if [ $1 = 0 ] ; then
/sbin/service %{name}-account stop >/dev/null 2>&1
/sbin/chkconfig --del %{name}-account
fi
%postun account
if [ "$1" -ge "1" ] ; then
/sbin/service %{name}-account condrestart >/dev/null 2>&1 || :
fi
%post container
/sbin/chkconfig --add %{name}-container
%preun container
if [ $1 = 0 ] ; then
/sbin/service %{name}-container stop >/dev/null 2>&1
/sbin/chkconfig --del %{name}-container
fi
%postun container
if [ "$1" -ge "1" ] ; then
/sbin/service %{name}-container condrestart >/dev/null 2>&1 || :
fi
%post object
/sbin/chkconfig --add %{name}-object
%preun object
if [ $1 = 0 ] ; then
/sbin/service %{name}-object stop >/dev/null 2>&1
/sbin/chkconfig --del %{name}-object
fi
%postun object
if [ "$1" -ge "1" ] ; then
/sbin/service %{name}-object condrestart >/dev/null 2>&1 || :
fi
%post proxy
/sbin/chkconfig --add %{name}-proxy
%preun proxy
if [ $1 = 0 ] ; then
/sbin/service %{name}-proxy stop >/dev/null 2>&1
/sbin/chkconfig --del %{name}-proxy
fi
%postun proxy
if [ "$1" -ge "1" ] ; then
/sbin/service %{name}-proxy condrestart >/dev/null 2>&1 || :
fi
%files
%defattr(-,root,root,-)
%doc AUTHORS LICENSE README
%doc etc/dispersion.conf-sample etc/drive-audit.conf-sample etc/object-expirer.conf-sample
%doc etc/swift.conf-sample
%dir %{_datarootdir}/%{name}/functions
%dir %attr(0755, swift, swift) %{_localstatedir}/run/swift
%dir %{_sysconfdir}/swift
%dir %{python_sitelib}/swift
%{_bindir}/swift
%{_bindir}/swift-account-audit
%{_bindir}/swift-bench
%{_bindir}/swift-drive-audit
%{_bindir}/swift-get-nodes
%{_bindir}/swift-init
%{_bindir}/swift-ring-builder
%{_bindir}/swift-dispersion-populate
%{_bindir}/swift-dispersion-report
%{_bindir}/swift-recon*
%{_bindir}/swift-object-expirer
%{_bindir}/swift-oldies
%{_bindir}/swift-orphans
%{_bindir}/swift-form-signature
%{_bindir}/swift-temp-url
%{python_sitelib}/swift/*.py*
%{python_sitelib}/swift/common
%{python_sitelib}/swift-%{version}-*.egg-info
%files account
%defattr(-,root,root,-)
%doc etc/account-server.conf-sample
%dir %{_initrddir}/%{name}-account
%dir %attr(0755, swift, swift) %{_localstatedir}/run/swift/account-server
%dir %{_sysconfdir}/swift/account-server
%{_bindir}/swift-account-auditor
%{_bindir}/swift-account-reaper
%{_bindir}/swift-account-replicator
%{_bindir}/swift-account-server
%{python_sitelib}/swift/account
%files container
%defattr(-,root,root,-)
%doc etc/container-server.conf-sample
%dir %{_initrddir}/%{name}-container
%dir %attr(0755, swift, swift) %{_localstatedir}/run/swift/container-server
%dir %{_sysconfdir}/swift/container-server
%{_bindir}/swift-container-auditor
%{_bindir}/swift-container-server
%{_bindir}/swift-container-replicator
%{_bindir}/swift-container-updater
%{_bindir}/swift-container-sync
%{python_sitelib}/swift/container
%files object
%defattr(-,root,root,-)
%doc etc/object-server.conf-sample etc/rsyncd.conf-sample
%dir %{_initrddir}/%{name}-object
%dir %attr(0755, swift, swift) %{_localstatedir}/run/swift/object-server
%dir %{_sysconfdir}/swift/object-server
%{_bindir}/swift-object-auditor
%{_bindir}/swift-object-info
%{_bindir}/swift-object-replicator
%{_bindir}/swift-object-server
%{_bindir}/swift-object-updater
%{python_sitelib}/swift/obj
%files proxy
%defattr(-,root,root,-)
%doc etc/proxy-server.conf-sample
%dir %{_initrddir}/%{name}-proxy
%dir %attr(0755, swift, swift) %{_localstatedir}/run/swift/proxy-server
%dir %{_sysconfdir}/swift/proxy-server
%{_bindir}/swift-proxy-server
%{python_sitelib}/swift/proxy
%files doc
%defattr(-,root,root,-)
%doc LICENSE
#%doc doc/build/html
%changelog
* Thu Apr 26 2012 Anthony Towns <atowns@redhat.com> 1.4.8-2
- Apply gluster patches
- Rename to gluster-swift
* Thu Mar 22 2012 Alan Pevec <apevec@redhat.com> 1.4.8-1
- Update to 1.4.8
* Fri Mar 09 2012 Alan Pevec <apevec@redhat.com> 1.4.7-1
- Update to 1.4.7
* Mon Feb 13 2012 Alan Pevec <apevec@redhat.com> 1.4.6-1
- Update to 1.4.6
* Thu Jan 12 2012 Alan Pevec <apevec@redhat.com> 1.4.4-2
- add back /var/run/swift for el6
* Wed Jan 04 2012 Alan Pevec <apevec@redhat.com> 1.4.4-1
- Use updated parallel install versions of epel packages (pbrady)
- Ensure the docs aren't built with the system glance module (pbrady)
- Ensure we don't access the net when building docs (pbrady)
- Update to 1.4.4
* Wed Nov 23 2011 David Nalley <david@gnsa.us> -1.4.3-2
* fixed some missing requires
* Sat Nov 05 2011 David Nalley <david@gnsa.us> - 1.4.3-1
- Update to 1.4.3
- fix init script add, registration, deletion BZ 685155
- fixing BR to facilitate epel6 building
* Tue Aug 23 2011 David Nalley <david@gnsa.us> - 1.4.0-2
- adding uid:gid for bz 732693
* Wed Jun 22 2011 David Nalley <david@gnsa.us> - 1.4.1-1
- Update to 1.4.0
- change the name of swift binary from st to swift
* Sat Jun 04 2011 David Nalley <david@gnsa.us> - 1.4.0-1
- Update to 1.4.0
* Fri May 20 2011 David Nalley <david@gnsa.us> - 1.3.0-1
- Update to 1.3.0
* Tue Feb 08 2011 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.1.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_15_Mass_Rebuild
* Sun Dec 05 2010 Silas Sewell <silas@sewell.ch> - 1.1.0-1
- Update to 1.1.0
* Sun Aug 08 2010 Silas Sewell <silas@sewell.ch> - 1.0.2-5
- Update for new Python macro guidelines
- Use dos2unix instead of sed
- Make gecos field more descriptive
* Wed Jul 28 2010 Silas Sewell <silas@sewell.ch> - 1.0.2-4
- Rename to openstack-swift
* Wed Jul 28 2010 Silas Sewell <silas@sewell.ch> - 1.0.2-3
- Fix return value in swift-functions
* Tue Jul 27 2010 Silas Sewell <silas@sewell.ch> - 1.0.2-2
- Add swift user
- Update init scripts
* Sun Jul 18 2010 Silas Sewell <silas@sewell.ch> - 1.0.2-1
- Initial build

View File

@ -220,14 +220,14 @@ class DiskDir(DiskCommon):
self.metadata[X_BYTES_USED] = int(self.metadata[X_BYTES_USED]) - int(content_length)
self.put_metadata(self.metadata)
def put_container(self, timestamp, object_count, bytes_used):
def put_container(self, container, put_timestamp, del_timestamp, object_count, bytes_used):
"""
For account server.
"""
self.metadata[X_OBJECTS_COUNT] = 0
self.metadata[X_BYTES_USED] = 0
self.metadata[X_CONTAINER_COUNT] = int(self.metadata[X_CONTAINER_COUNT]) + 1
self.metadata[X_PUT_TIMESTAMP] = timestamp
self.metadata[X_PUT_TIMESTAMP] = 1
self.put_metadata(self.metadata)
def delete_container(self, object_count, bytes_used):

View File

@ -15,6 +15,8 @@
import os
from eventlet import tpool
from tempfile import mkstemp
from contextlib import contextmanager
from swift.common.utils import normalize_timestamp, renamer
from swift.plugins.utils import mkdirs, rmdirs, validate_object, \
check_valid_account, create_object_metadata, do_open, \
@ -294,3 +296,21 @@ class Gluster_DiskFile(DiskFile):
if X_OBJECT_TYPE in self.metadata:
self.metadata.pop(X_OBJECT_TYPE)
@contextmanager
def mkstemp(self):
"""Contextmanager to make a temporary file."""
if not os.path.exists(self.tmpdir):
mkdirs(self.tmpdir)
fd, tmppath = mkstemp(dir=self.tmpdir)
try:
yield fd, tmppath
finally:
try:
os.close(fd)
except OSError:
pass
try:
os.unlink(tmppath)
except OSError:
pass

View File

@ -67,7 +67,6 @@ class Glusterfs(object):
def get_export_list_remote(self):
export_list = []
cmnd = 'ssh %s gluster volume info' % self.mount_ip
print 'Remote'
if os.system(cmnd + ' >> /dev/null'):
raise Exception('Getting volume info failed %s, make sure to have \

View File

@ -13,10 +13,6 @@ account_autocreate = true
[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3
[filter:healthcheck]
use = egg:swift#healthcheck

View File

@ -323,33 +323,21 @@ def check_user_xattr(path):
def _check_valid_account(account, fs_object):
mount_path = getattr(fs_object, 'mount_path', MOUNT_PATH)
if not check_account_exists(fs_object.get_export_from_account_id(account), \
fs_object):
if os.path.ismount(os.path.join(mount_path, account)):
return True
if not check_account_exists(fs_object.get_export_from_account_id(account), fs_object):
logging.error('Account not present %s', account)
return False
if not os.path.ismount(os.path.join(mount_path, account)):
if not os.path.isdir(os.path.join(mount_path, account)):
mkdirs(os.path.join(mount_path, account))
fs_object.unmount(os.path.join(mount_path, account))
if fs_object:
if not fs_object.mount(account):
return False
if not check_user_xattr(os.path.join(mount_path, account)):
logging.error('Error: No support for user.xattr on backend %s' % account)
return False
chmod_cmd = ['chmod 777 %s' % (mount_path), \
'chmod 777 %s/%s' % (mount_path, account)]
for cmd in chmod_cmd:
if os.system(cmd):
logging.error('Chmod failed: %s' % (cmd))
return False
return True
def check_valid_account(account, fs_object):

File diff suppressed because it is too large Load Diff