Currently queues are lazy on operations with messages, but not with
subscriptions. It means the user is forced to pre-create a queue before
creating a subscription to the queue, which is not very convenient.
Also currently if a queue has a subscription, the subscription will
stay on even if the user will delete the queue. But even if so Zaqar
will still work normally. This is strange that subscriptions can exist
with deleted corresponding queues, but it's impossible to create a
subscription to yet unexising queue.
This patch makes queues lazy also on operations with subscriptions, so
the user will be able to subscribe to yet unexisting queue.
Also this patch modifies tests, to make sure Zaqar's subscriptions
work both with pre-created queues and non-existing queues.
APIImpact
DocImpact
Implements: blueprint lazy-queues-in-subscriptions
Change-Id: I814b503243c4e06e74acc6b709bda4269df889e9
oslo.utils method is_valid_ipv6 catches AddrFormatError exception
in case of invalid ipv6 address. This method is warpper around
netaddr.valid_ipv6 method, and ensure always return boolean value.
Change-Id: Idead014cc53600f599810b849789ea67a3a3f1bf
Now pipeline is heavily used in Zaqar for controller, but unfortunately
operators will see a lot of WARNING errors in Zaqar's log because
some of the controllers don't implement one of the methods. That should
be a development details and it's not necessary to expose as WARNING,
otherwise, operators will be scared.
Change-Id: I98c574dc240372627d9e760fc0253c6340a9463f
Query for all subscriptions on a given queue by taking into account the
returned marker, if any.
Change-Id: I6b03dd93cabc76d1c91041a896fbb30ac08032f1
Closes-Bug: #1590173
When creating a claim for messages in a queue, after
this claim expires, Zaqar will return service unavailable
when getting the claim.
This happens since claim's id is still listed in set
"<project_id>.<queue_name>.claims" but Zaqar get '[None]'
from claim_id's 'expire' field.
Fix this issue if field value is '[None]' then just return
it and remove it from set "<project_id>.<queue_name>.claims".
Change-Id: I8602b197903acc1692185e7715a4070814fe1661
Closes-Bug: #1588247
Make the max_workers attribute of NotifierDriver configurable.
Now it can be configured using 'max_workers' parameter. Default
value of max_workers is 10.
Co-Authored-By: wangxiyuan<wangxiyuan@huawei.com>
Change-Id: Ice7d12199f8463793951932b48408e7df81440f2
We recently fixed subscriptions and now they are expirable resources as
they should be.
This patch adds 'age' field to subscription resource representation
which is exposed to the user and defined in redis driver.
This patch also removes existing 'expires' field, because:
1. The 'expires' field is missing in mongodb's subscription resource
representation. (But age field was also missing until a recent patch).
2. The 'expires' field has int (timestamp) format, which is not human
readable.
3. Other Zaqar's expirable resources show 'age' and 'ttl' fields instead
of showing 'expires' field.
4. There is no need in 'expires' field when 'age' and 'ttl' fields are
exposed to the user.
This change in unlikely to affect backward compatibility, because
subscription expirations started to work only recently.
Closes-Bug: 1552845
APIImpact
DocImpact
Change-Id: I18f908b2f137bd4b8e5b3d3275063a1d37942093
We recently fixed subscriptions and now they are expirable resources as
they should be.
But now, unlike messages and claims, they don't show their age, so it's
impossible for the user to know when the particular subscription will
expire.
This patch adds 'age' field to subscription resource representation
returned from mongodb driver. Tests are modified to make them check also
'age' field.
Partial-Bug: 1552845
APIImpact
DocImpact
Change-Id: I5eb6a6a267161ec7ee6a7cda094dd55960520069
Now,Zaqar will always return "max_messages_post_size"
and "default_message_ttl" when get a queue's metadata.
So users can't know whether these values are in db or
not. Then if they take the "replace" or "remove" action
to update the queue's metadata when these two values are
not in db, they will get an error: "can not update(remove)
non-exist metadata".
So we should check them before the queue update action.
Change-Id: Ia97e80a406c8321c7e8d0aa2652023049e5ee4c0
Closes-bug: #1570160
Now Zaqar isn't using Tempest for jenkins gate checking, which is
a missing. This patch fixes it by using Tempest's all-plugin to
run Zaqar's Tempest cases in tree.
Change-Id: I7e10a5b218b60eea47a96d36ebeb4029d33e7edd
Now zaqar's sql backend always use only one connection
to db. It will lead an multi-thread problem:
pymysql.err.InternalError) Packet sequence number wrong
So we should use session in sqlalchemy to avoid this happen.
Change-Id: I84e2f9899e141aed151f7c06fc4b1f603481886e
Closes-bug: #1588117
If some messages in Redis are already expired, it's impossible to claim
any new messages.
Zaqar internally catches this error from running "claim_messages.lua":
"ResponseError: Error running script: @user_script:59: user_script:59:
attempt to compare nil with number". And returns 503 response code to
the user.
This happens because if some message is expired (it's <message_id>
record was automatically deleted from database), it's ID is still listed
in set "<project_id>.<queue_name>.messages" and Zaqar uses this ID. In
this case on attempt to get some values from <message_id>'s fields, it
gets 'nil' values.
This patch makes Zaqar check if returned field values from <message_id>
record are 'nil'. If values are not 'nil', process the record normally.
If values are 'nil', remember message ID. In the end of the script
"claim_messages.lua" garbage collect all such message IDs, because it's
a good opportunity to do so.
Closes-Bug: 1548135
Change-Id: Iedd2b82f6da30dd38bfdbb837bf9d0d4c282e222
This patches adds the basic architecture to adopt api ref in tree
and a new tox command to build api ref for testing.
A new api ref gate job will be added to verify the format which
will be tracked by another commit against project-config.
blueprint: api-ref-in-tree
Change-Id: Idffbdc9325868a3dcd6a9bd7f90f030fd52ee0dc
In f383bd2973c58c5272504603e4f9dddd55c5bb68, oslo.context (2.2.0)
added roles support in the context itself. This passes roles to
super()__init__.
Change-Id: I0535728846006c0ad26c75a8fe36071e96e96f71
The sqlalchemy driver uses a run() wrapper instead of directly
using the connection.execute() method. I changed the other use
of connection.execute() to run() for consistency.
Closes-Bug: #1469207
Change-Id: I0cfe043f31e8bebe26c983c734e0db754e122fbe
When notification pipeline and pooling are both enabled, the method
'post' of NotifierDriver will be called two times. One for normal
SubscriptionController and one for pooling.SubscriptionController.
So a check was added to avoid the notifition is sent two times, but
unfortunately an unintentional error was introduced. This patch fixes
it by refactoring the condition checking.
Change-Id: Ie22748e86a50b7d91f24707b0c5a3ab718f005a2
We're using zaqar.notification.tasks to look for the notifier
driver by stevedore, so this patch just rename the package
name to keep consistence. No impact for end user and migration.
Change-Id: I7694af4dbc4f73703e429560fb0a1457f3e1193a
Since tempest-lib is deprecated for future bug-fixes and code changes
in favor of tempest, so Zaqar's tempest code need to be changed
accordingly. See
http://docs.openstack.org/developer/tempest-lib/readme.html
Change-Id: I095e8ea3fe09fa9b4e81c6a25abe200b5d5fcd15
Refactor the code merged to fix the py34 issue,
let it follow the DRY principle.
Depends-on: I6da301dc4935ec8e4254d71ca93036f3d8db2bcf
Change-Id: Iaa0768e795e172649787dc6d26722848b5d732ee
Closes-Bug: #1584958
This option is deprecated in Mitaka and has been removed in Newton,
and its default value is True already. Let's remove it.
Change-Id: I167713165bd96a4ccd423976f8fcad5ff8fc399c
ref: https://review.openstack.org/#/c/314573/
Since hook feature has removed after 1.0.0, using
middleware instead of it, but for the compatibility
with old version, we support them both now.
Closes-Bug: #1583358
Change-Id: Ie182017d982efd727c63634b2a168dadd41711e4
Add a configuration wrapper to avoid reentrant call to __getattr__()
in copy.copy(conf) on Python 3.x which is causing Zaqar's py34 gate job
failure randomly. The code is copied from Cinder, see the similar fix
in Cinder: https://review.openstack.org/#/c/280241/
Co-Authored-By: wangxiyuan<wangxiyuan@huawei.com>
Co-Authored-By: Wang Hao<wanghao749@huawei.com>
Change-Id: I627b96f00e5b05993b8f4f586ea216214d5f1a5f
Instead of using the now removed hook feature of falcon, let's use
the middleware feature.
NOTE: Though the test is changed. But actually it's not correct since
the unicode test with a running zaqar server will return 404 instead
of 400. But it's out of the scope of this fix. Will fix it in a
following patch.
Co-Authored-By: Fei Long Wang <flwang@catalyst.net.nz>
Closes-Bug: #1581189
Change-Id: I7e8e97b73ec3e245580f07e62442caf0395d5745
