I use python uwsgi to make my code. there is a error occur sometime when i send message to rabbbitmq and didn't use confirm mode. the error like this:
rabbmitmq server log error =ERROR REPORT==== 21-Jul-2022::15:23:04 === Error on AMQP connection <0.23590.991> (172.198.12.10:59211 -> 10.0.12.1:5672, vhost: 'host', user: 'host', state: running), channel 12848: operation none caused a connection exception frame_error: "type 91, all octets = <<>>: {frame_too_large,842149168,131064}"
in other way, i find my python create log file fd also put some error logs :
python log error IOError: [Errno 9] Bad file descriptor Traceback (most recent call last): File "/usr/lib64/python2.6/logging/init.py", line 800, in emit self.flush() File "/usr/lib64/python2.6/logging/init.py", line 762, in flush self.stream.flush()
this is create python log file code :
def init_logger(self):
self._logger = logging.getLogger(self._proj)
self._logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('[%(asctime)s] [%(process)d] [%(levelname)s] %(message)s')
if not self._b_stream_init:
stream_handler = logging.StreamHandler(sys.stderr)
stream_handler.setFormatter(formatter)
stream_handler.setLevel(logging.DEBUG)
self._logger.addHandler(stream_handler)
self._b_stream_init = True
ret = self.check_log_name()
if ret[0]:
return 0
try:
log_file_handler = logging.FileHandler(ret[1])
log_file_handler.setFormatter(formatter)
log_file_handler.setLevel(CLog.LEVEL_MAP[self._log_level])
self._logger.addHandler(log_file_handler)
if self._last_file_handle is not None:
self._logger.removeHandler(self._last_file_handle)
self._last_file_handle.close()
self._last_file_handle = log_file_handler
self._last_log_name = ret[1]
except:
pass
def check_log_name(self):
if self._log_dir is None or self._log_prefix is None:
return True, None
log_name_arr = [self._log_dir, self._log_prefix, '_', time.strftime('%Y%m%d_%H'), '.log']
log_name = ''.join(log_name_arr)
if self._last_log_name != log_name or not os.path.exists(log_name):
return False, log_name
else:
return True, log_name
this code raise exception but i can't find out the reason why.
this code i use client send message to rabbitmq server:
@trace_report(switch=True)
def send(self, key, message, declare=False, expiration=None):
if declare:
self._declare_exchange()
self._declare_queue_with_key(key)
if isinstance(expiration, int):
expiration = str(expiration)
properties = pika.BasicProperties(delivery_mode=2,
expiration=expiration)
self.channel.basic_publish(exchange=self.exchange, routing_key=key,
body=message, properties=properties)
I guess that the file system write log content to the rabbitmq server socket fd by mistake. what i can do in this situation.
ps: socket_wait was so much(4w) but without no any kernel log
init_logger
to trace what's going on. Somehow, your logging file handle is getting closed. Do ANY log messages work? Do you know what check_log_name
returns? {frame_too_large,842149168,131064}
means that you sent a message larger than what the RabbitMQ server is configured for.
You can try increasing the max-frame configuration of the server but it can impact performance of it as well.