openstack/monasca-common
Code Issues Proposed changes

Catch the case when no new messages are available

Browse Source 
Confluent Kafka client throws the KafkaError._PARTITION_EOF exception
when no new messages are available in a given partitiion. We should
catch this case and continue consuming after a short sleep.

Story: 2003705
Task: 30116
Change-Id: I44add24df764bbc4e718358a8af75903e035f3f0
This commit is contained in:
Witek Bedyk 2019-03-19 09:46:15 +01:00
parent 5cfcbda66c
commit 5bdd98662b
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
monasca_common/confluent_kafka/consumer.py
View File

@ -70,6 +70,10 @@ class KafkaConsumer(object):
continue continue
elif not message.error(): elif not message.error():
yield message yield message
elif message.error().code() == \
confluent_kafka.KafkaError._PARTITION_EOF:
time.sleep(0.1)
continue
else: else:
log.error("Kafka error: %s", message.error().str()) log.error("Kafka error: %s", message.error().str())
raise confluent_kafka.KafkaException(message.error()) raise confluent_kafka.KafkaException(message.error())