This is a manual forward-port of the following change merged into
icehouse branch: https://review.openstack.org/215119
When content of an object is modified from filesystem interface, a GET
on the object will return inconsistent or incomplete content because the
content length originally stored as metadata no longer reflects the
actual length of the file after modification.
The complete fix will have two parts:
(1) Return the entire content of object as is to the client
(2) The Etag returned should reflect the actual md5sum of object content
This change only fixes (1) mentioned above. This means, the client will
always get the complete content of the file.
Fix (2) is not part of this change. This means, if content length of the
object remains same even after modification, the Etag returned would be
incorrect. Fixing (2) involves more invasive changes in the code. So
that is deferred for now and will be sent as a separate change later.
Reference:
https://bugs.launchpad.net/swiftonfile/+bug/1416720https://review.openstack.org/151897
Change-Id: I28d0ec33c59eb520be7d15a60adb968692226e3e
Closes-Bug: #1416720
Signed-off-by: Prashanth Pai <ppai@redhat.com>
This is a manual forward-port if the following change in icehouse
branch: https://review.openstack.org/211866
Object server never closes file descriptor when DiskFile.open()
raises an exception. This is fixed by catching exceptions thrown
by code block inside DiskFile.open() and manually closing the
file descriptor.
Change-Id: Ie3e047443f4893e21b9cbdea691867f069363d7e
Signed-off-by: Prashanth Pai <ppai@redhat.com>
The xattr module raises IOError which is what the call to
read_metadata() in rmobjdir() should be catching.
Change-Id: I2983ad0be453647f80b862eea93a5fc16fcef04a
Signed-off-by: Prashanth Pai <ppai@redhat.com>
restrict pyeclib version to 1.0.7, similar to swift
Change-Id: Ia69b6e1ac82a60d5cca9bbac8bb355124dc75cd6
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Signed-off-by: Prashanth Pai <ppai@redhat.com>
The intent was (and still is) to devise an unique name for the tempfile.
I can't think of any reason why uuid wouldn't work.
Also, added a comment which has a link to the document explaining why
the specific naming convention is used for temp file name.
Change-Id: I5f0d2b631e276c47c88c554485caaec767703695
Signed-off-by: Prashanth Pai <ppai@redhat.com>
These are old remnant code that is no longer in use.
Change-Id: Ic0afe1f16884efec49bfa2f64345129e1421daa6
Signed-off-by: Prashanth Pai <ppai@redhat.com>
This change does the following:
* Update all functional tests, unit tests and conf files based on
Swift 2.3.0 release.
* Fix "test_versioning_dlo" test from Swift to be SOF compatible.
* Bump swiftonfile release version to 2.3.0
* Update swiftonfile with DiskFile API changes
Change-Id: I75e322724efa4b946007b6d2786d92f73d5ae313
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Add IOERROR.strerror to exception messages in utils.py
Correct some types in exception messages in fs_utils.py
Change-Id: Iac81860b08daf8cdbb7f8e8950f4ab6104b75bd4
Closes-Bug: #1445237
Allow a slash at the end of object file name but ONLY if the PUT
request also has a "Content-type: application/directory" header.
Change-Id: Ic775de052ee3635e95f5e32ca6e2038909fe1005
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Override Swift's REPLICATE request handler method with a no-op which
returns HTTPNotImplemented (501).
Change-Id: Iaa52ca7e66ddaeda55e73841cd06b64e3e8a3369
Signed-off-by: Prashanth Pai <ppai@redhat.com>
NOTE:
The previous rebase was to Swift 2.1.0 and this rebase is to
Swift 2.2.1 (first release in kilo series). There was a
Swift 2.2.0 (last release in juno series) release in between.
Change-Id: Ibce2e299935e165db89a91a6fe8c4c5c027db098
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Add note about swift geo-replication not being supported for
Swift on File containers.
Also add reference and links to Swift on File presentation from
Paris summit.
Change-Id: If8b4608d7b20c16b4e8890728eb9714db01becc9
Used vulture utility to detect unused code in swiftonfile.
https://pypi.python.org/pypi/vulture
Change-Id: I045f5a96a48d2384718ad0f993540caa3a866309
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Current tree does not collect everything available when the usual
"python setup.py sdist" is run. Resulting tarball works, but cannot
be used to build complete RPMs.
Strangely puzzling, adding include_package_data=True does not work.
It simply has no effect. So, we add MANIFEST.in like in the old days.
Since we're at it, add everything else missing, like the README.md.
Stock setuptools in F21 only collects README.txt, but not Markdown.
Change-Id: I9fb89429a8b690fc7e8e128962aeb991a1f760d9
Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.
Change-Id: Id6a13d0396e16e0e9e1a08f3d7d24566d8ccccb5
SOF can support object names of upto 1024 if it conforms to the
following constraints:
Object names can have forward slashes ('/') in them. Each segment in
between these slashes cannot exceed 255 characters with exception of
the last segment which cannot exceed 221 characters.
This constraint arises from the fact that each segment except the last
one in object name is a directory, the last segment being an actual
file on the filesystem.
Also, restored default constraint values in swift.conf since the SOF
constraints middleware (always in proxy pipeline) will take care of it.
Change-Id: Ia7dc44671a87911c092fecd0344eace92f5c225b
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Adding a constraints middleware that allows different constraints for a
Swift-on-File storage policy.
Currently, Swift allows only one set of object naming and metadata rules
per cluster. As new storage policies are implemented, it is possible that
different storage back-ends have different set of rules on how objects should
be created for that specific storage system. Swift-on-File has different rules
as objects name are mapped to a POSIX filesystem directory structure.
For example: no names with double slashes or ending with slashes are allowed.
At first a solution was proposed to include a generic patch in the upstream
swift code, but after discussing with the Swift community it became clear that
it would be better to have a global set of constraints that covers the whole
cluster and if a specific storage policy has specific constraints than it
should have its own middleware.
Link to patch for reference: https://review.openstack.org/#/c/113325/
Change-Id: I323ead5d98bf5c087930ccf446d3e8d83075e584
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Updated sof version and tox swift dependency to
release 2.0.
Also, made some documentation changes to the object-server
configuration sample file to point out that volume must
mounted before starting sof object-server.
Signed-off-by: Thiago da Silva <thiago@redhat.com>
