r software for log file analysis
Although I selected Apache log files to analyze, you can get your data from anywhere you can imagine including Squid log files and uptime, sar, or vmstat output. In case you are not familiar with R , it is a GNU project based on S, which is a statistics specific language and environment developed at the famous ATT Bell Labs. I am using R primarily because it can handle huge amounts of data. I am not so sure that Microsoft Excel, for example, could process 1 million lines of data. You may find other software that can automatically analyze Apache log files and create reports, but it is easier to generate your own customized reports by using R. And, as you are going to see later, R can deal with log files pretty easily. The first time you try a new R command, it is better to use the R interactive shell, but keep in mind that all R commands can be put into an R script and be executed from the UNIX shell or as a cron job. I use RStudio for interacting with the R shell but you can also run R on its own. Nevertheless, RStudio is free, and my advice is to use it as it simplifies your interaction with R.
FREE IEEE PAPER