Friday, November 24, 2006

Linux tip of the day

If you have a busy server, check how big a log file is before trying to open it with your favourite light weight text editor.

Made this mistake today.

One of the mail servers I run for a client gets some email, but gets an horrific amount of spam, and attempts at spam. In fact it looks almost like a concerted denial of service attempt.

I greped the logfile to see how many noqueues it had. 164,000 in 5 days. That is just silly. I then tried to open the logfile to see how many lines it had in it and lost the server.

I almost got in far enough to kill the process but not quick enough. The server had to be rebooted.

The mail log was over 1GB in just 5 days. This mail server only handles one domain.

Just how are ISP's supposed to keep 6 months records of emails? They manage thousands or hundreds of thousands of domains. The log files must grow at a huge rate.

9 Comments:

Blogger dizzy said...

cat $file | wc -l

I suggest logrotate, gzip and cron. You could write a logrotation script in about ten minutes really.

Just fire a script off at 1 minute to mindight. Chuck a sleep in it for 61 second and then move your sendmail log to a new name with a date appended. Touch out a new sendmail log file and kill -HUP sendmail, then gzip the file up, job done.

Obviously works the same for postfix if you're using a lesser mail server :)

2:49 pm  
Blogger Benedict White said...

Actually the system already logrotates weekly, which WAS fine. Now I obviously need to change that to daily which I will do on Monday.

I note you wish to start a religious war. Postfix is better so there!

Actualy I used Sendmail once, didn't care for it. I like how flexible and easy to use postfix is, and I get on with Wietse on the mailing list which is handy

3:10 pm  
Blogger dizzy said...

postfix doesn't have a genuine listening daemon and effectively elevates a pid everytime mail calls are made to the daemon. In an large ISP environment it would b0rk horribly if it was running as an MX server.

It's fine for SMEs, and it's config file is nicer, but sendmail is the beast for large scale mail server farms (or Exim). It's far more extensible too.

5:27 pm  
Blogger Benedict White said...

Fair enough Dizzy, that answers the question. I wonder if Weitse is thinking of changing that?

I will ask him.

Out of interest how many mails a day do your servers handle?

One of the best servers for handling messages I have seen is INN, but that is not a mail server.

5:59 pm  
Blogger dizzy said...

how many? Billions.

11:44 am  
Blogger dizzy said...

having said that I couldn't tell you if they;re suing sendmail or not. I've just been reliably informed that Claranet uses Postfix. Best of my knoweldge, Pipex (which encompasses GXN) uses Sendmail. Tiscali use CriticalPath (which sucks).

11:47 am  
Blogger Benedict White said...

I see. It would be interesting to know what mta handles billions a day.

See postfix ain't that bad ;)

I must also remember to write about NPTB errors. (Similar but different to NPI errors)

11:56 am  
Blogger T-T-Tommy said...

This is COMPLETELY off topic, but I see where you got your template from. You commented on hackosphere about it.. and it's beautiful work. it's almost exactly what I'm trying to do with mine, but my coding days are sadly beyond me. Would you mind sharing your template XML code with me? I'll gladly give you credit for it and would of course change it so it doesn't look like yours. Great work with it.

6:13 am  
Blogger Benedict White said...

Mithrandir (Gandalf), The template for this blog is a straight forward beta blog minima with no modifications.

I use a 3 column template at my political blog here:
http://aconservatives.blogspot.com/

which has one tiny mod to fix an IE issue and also here:
http://feedingtheblog.blogspot.com/

which is the one I downloaded from hackosphere.

If you want to use the one on my conservative blog, just download the hackosphere one, and I will tell you the line you need to alter to fix it for IE.

12:22 pm  

Post a Comment

<< Home