Fri, 23 Oct 2009 10:35:12 -0400 On Friday 23 October 2009, Michael S. Zick wrote: > On Fri October 23 2009, Chuck wrote: > > On Friday 23 October 2009, Edward Capriolo wrote: > > > On Fri, Oct 23, 2009 at 6:58 AM, Chuck <chuck@sbbsnet.net> wrote: > > > > On Friday 23 October 2009, Konrad Korzeniowski wrote: > > > >> Hi, > > > >> > > > >> I don't know syslog-ng, but there is rsyslog solution: > > > >> http://www.rsyslog.com/doc-rsyslog reliable forwarding.html > > > >> > > > > > > > > thanks. i will look into it. > > > > > > > >> rgrds., > > > >> KK > > > >> > > > >> [2009-10-23 04:16] Chuck: > > > >> > I doubt this is a vserver problem, but I am placing this here just in > > > > case > > > >> > there may be something. > > > >> > > > > >> > we run a remote logging server that every host and guest logs to in > > > > addition > > > >> > to local logs. when all is well this works just fine. > > > >> > > > > >> > if the remote log server is taken down, suddenly most of the hosts > > begin > > > >> > chewing up all available memory and swap until they crash. disabling > > > > remote > > > >> > logging on everything cured this but obviously is not what we want. > > > >> > > > > >> > i am also posting this in the gentoo forum in case it is a syslog-ng or > > > > other > > > >> > issue. > > > >> > > > > >> > we use 2 lines in every syslog-ng config.. obviously the ip is changed > > for > > > >> > public viewing: > > > >> > > > > >> > destination remote { udp("1.3.0.1"); }; > > > >> > log { source(src); destination(remote); }; > > > >> > > > > >> > on all machines > > > >> > kernel is > > > >> > > > > >> > 2.6.26-vs2.3.0.35.6-gentoo > > > >> > > > > >> > util-vserver is 0.30.215 > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > > > > > > > > > > > > > -- > > > > > > > > Chuck > > > > > > > > > > Chuck, > > > > > > Strange, since UDP packets have no guarantees for transmission or > > > delivery one would imagine they would be sent and dropped. Maybe > > > syslog-ng has a delivery queue that that holds messages until it can > > > make a connection to the remote server and since your server is down > > > the messages build up. > > > > > > > after reading about the rsyslog that suddenly occurred to me too... i am going > > to look into the options available for the config to see if there is a queue > > and if so maybe it can be disabled. i always thought udp was 'blind' and that > > syslog-ng would just blindly send out and if it reached the destination fine > > if not it would not know the difference and continue on.. evidently not.. > > > > UDP is 'blind' in the sense that it does not provide guarantied delivery. > > What may be happening here is a failure *before* the packet is sent - > I.E: Your routing is returning "Network unreachable" - so the packet > has nowhere to go. > > Perhaps you could provide an alternate route to a packet "/dev/null" > (I.E: the "dummy" interface) for when the for-real logger is down. > I think the stock network bridging module can be set-up to do this for you. > i think i found the answer in syslog changelog: 2.1.4 Tue, 17 Mar 2009 13:35:00 +0100 Bugfixes: * Fixed a possible DoS condition triggered by a destination port unreaachable ICMP packet received from a UDP destination. syslog-ng started eating all available memory and CPU until it crashed if this happened. * Fixed the rate at which files regular were read using the file() source. * Report connection breaks as a write error instead of reporting POLLERR as the write error path reports more sensible information in the logs. > Mike > > > Edward > > > > > > > > > > > > -- Chuck