1

We are trying to write a script "sendemail.sh" to count the number of occurrence of a particular string in a log file "SendEmail.log" within the given interval. We have a log file. In that we are searching for a pattern "ReqInputMsgLog" and need to count the number of times it occurred in the given period for eg: from "2014-08-19 11:30" to "2014-08-19 11:34". And our script look like this:

#!/bin/sh
enterdate=$1
echo $enterdate
enddate=$2
enterdate1=`date +%s -d $enterdate +"%Y-%m-%d %H:%M"`

echo $enterdate1
enddate1=`date +%s -d $enddate +"%Y-%m-%d %H:%M"`
echo $enddate
count=0
cat SendEmail.log | grep "ReqInputMsgLog" | awk -F "[" '{print $3}' | awk -F "," '{print $1}' > /con/scripts_server/file.txt
for line in `cat /con/scripts_server/file.txt`
do
logdate=`echo $line | awk -F : '{print $1":"$2}'`
if [[ $logdate < $enddate1 ]];
        then
        count=`expr $count + 1`
        fi
done
echo $count

But when we are trying to execute the script by the below command its not showing the proper count.

./sendemail.sh "2014-08-19 11:30" "2014-08-19 11:34"

Log file is very big one. Small chunk has been posted here.

INFO [SIBJMSRAThreadPool : 5] [2014-08-19 11:18:24,471] SendEmail - 8/19/14 11:18 AM,ECCF25B0-0147-4000-E000-1B830A3C05A9,ReqInputMsgLog,SendEmail,<?xml version="1.0" encoding="UTF-8"?>
<in:sendEmailRequestMsg xmlns:in="http://EmailMed/EmailMedInterface" xmlns:ns0="wsdl.http://EmailMed/EmailMedInterface" xmlns:ns1="http://EmailMed/EmailMedInterface" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:me="wsdl.http://EmailMed/EmailMedInterface" xsi:type="me:sendEmailRequestMsg">
<in:sendEmail xmlns:xci0="http://EmailMed/EmailMedInterface">
INFO [SIBJMSRAThreadPool : 7] [2014-08-19 11:18:14,235] SendEmail - 8/19/14 11:18 AM,ECCEFDB2-0147-4000-E000-1B830A3C05A9,ReqInputMsgLog,SendEmail,<?xml version="1.0" encoding="UTF-8"?>
<in:sendEmailRequestMsg xmlns:in="http://EmailMed/EmailMedInterface" xmlns:ns0="wsdl.http://EmailMed/EmailMedInterface" xmlns:ns1="http://EmailMed/EmailMedInterface" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:me="wsdl.http://EmailMed/EmailMedInterface" xsi:type="me:sendEmailRequestMsg">
<in:sendEmail xmlns:xci0="http://EmailMed/EmailMedInterface">
INFO [SIBJMSRAThreadPool : 7] [2014-08-19 11:18:14,241] SendEmail - xmlText: <?xml version="1.0" encoding="UTF-8"?>

after awk command we will get a file "/con/scripts_server/file.txt" which looks similar like below:

2014-08-19 11:28:03
2014-08-19 11:28:06
2014-08-19 11:28:17
2014-08-19 11:28:53
2014-08-19 11:29:02
2014-08-19 11:29:47
2014-08-19 11:29:57
2014-08-19 11:30:07
2014-08-19 11:30:17
2014-08-19 11:30:19
2014-08-19 11:30:19
2014-08-19 11:30:22
2014-08-19 11:30:25
2014-08-19 11:30:25
2014-08-19 11:30:36
2014-08-19 11:30:51
2014-08-19 11:30:56
2014-08-19 11:30:59
2014-08-19 11:30:59
2014-08-19 11:31:08
2014-08-19 11:31:25
2014-08-19 11:32:19
2014-08-19 11:32:22
2014-08-19 11:32:27
2014-08-19 11:32:28
2014-08-19 11:32:41
2014-08-19 11:32:49
2014-08-19 11:32:59
2014-08-19 11:33:27
2014-08-19 11:33:41
2014-08-19 11:34:07
2014-08-19 11:34:14
2014-08-19 11:34:21
2014-08-19 11:34:25
2014-08-19 11:34:38
2014-08-19 11:34:50
2014-08-19 11:34:58
pruthvi
  • 171

2 Answers2

1

Use the following to calculate the lines between two time variables. Put the following code in a file called countOcurrences.

#!/bin/bash

 awk "/$1/,/$2/"'{count++} END{ printf "There are %s lines\n",  count}' con/scripts_server/file.txt

Run it as follow.

./countOcurrences "2014-08-19 11:30:07" "2014-08-19 11:34:07"

If file.txt is filled with a new date/time each time a pattern match occurs, then the above will work.

Valentin Bajrami
  • 4,385
1

First of all I encountered 2 errors trying to recreate your problem date: extra operand 11:34:14' Trydate --help' for more information. ./script.sh: line 15: 1408448098: No such file or directory

From what I understand you isolated the dates which interest you into file.txt according to user input and you want to count occurrences in there.

I coded this:

#!/bin/bash

#Start/End dates to encolse count range
startDate="2014-08-19 11:28:00"
endDate="2014-08-19 11:35:00"
#Concert these dates to seconds since Epoch
startDateEpoch=$(date --date="$startDate" +%s)
endDateEpoch=$(date --date="$endDate" +%s)

#Read file.txt count occurences
while read line
do

processingDate=$(date --date="$line" +%s)
  if [ $processingDate -lt $endDateEpoch ] && \
  [ $processingDate -gt $startDateEpoch ]; then
    echo "APOEL FC";
  fi

done < file.txt

Your file.txt as appended here contains 37 lines, so:

sysadmin@omg:/tmp$ ./script.sh | wc
     37      74     333

This looks correct for

startDate="2014-08-19 11:28:00"
endDate="2014-08-19 11:35:00"

Changing the dates to: 

startDate="2014-08-19 11:28:03"
endDate="2014-08-19 11:34:58"

sysadmin@omg:/tmp$ ./script.sh  | wc
     35      70     315

returns 35 occurrences which looks correct since first and last dates should be excluded.

So, the conversion to seconds since Epoch must have had something wrong in the syntax and the < operator in your if statement was causing a small mess.

As general guidelines prefer to use -lt -gt (less than, greater than) operators for if. Also try to avoid enclosing command in backticks - Prefer to use $(command).

Happy coding.

Louis Papaloizou
  • 463