Homer bores me to tears... (How to monitor sip in realtime without homer)

Status
Not open for further replies.

Ian Oakes

Member
Oct 14, 2016
51
9
8
44
Hey All!


So as the title says, I am not real fond of homer. I suppose I could have spent more time with it and worked out all my grief, but ultimately its one or two more services and ips i have to devote to it. I can get what I want in a much simpler, and for me, faster way.

So, the trick here is using sngrep. If you haven't gotten acquainted, I would suggest doing so. It comes standard as part of the fusionpbx install now, which is cool!

Sngrep will show you a list of calls taken while its running, but like wireshark, tshark and homer, it will also read packet captures and show you the same thing. As i mentioned this can be used in near real time as sngrep can open running capture files, and give you calls up to the moment you opened it.

First, I am assuming sngrep is already installed, but if this is an older instance, follow the instructions here for package-based distros:
https://github.com/irontec/sngrep/wiki/Installing-Binaries

Alrighty, we are all installed, you are also going to need to install tcpdump and ngrep for fun, too.

Now, with that finished, you will want to run the following script:
Code:
#Full capture with media/rtp
#tcpdump -nq -s 0 -i eth0 -G1800 -w Sip_w_RTP_%Y-%m-%d_%H:%M:%S.pcap port 5060 or port 5080 or udp portrange 16384-32767 &

#Sip capture on 5060 and 5080
tcpdump -nq -s 0 -i eth0 -G1800 -w /root/captures/Sip_Only_%Y-%m-%d_%H:%M:%S.pcap port 5060 or port 5080 &

The two lines in there are for captures with media and without. Sometimes (rarely) you need to diagnose audio issues, so the first tcpdump line does that. You may have to modify the device, the ports, and the rtp range depending on whether your carrier sends on non-standard ports. I now use bash sngrep-capture.sh and it runs in the background. the -G flag specifies the interval to save files in this case its in 30 minute (1800 second) intervals to keep files manageable.

It will create the captures in the directory you specify, so watch the file path. You will want to leave this running all the time, so you will need to delete the files, which will eat up tons of space if you’re not careful. 3 days is a good range and I only ever have under a 100 gig of captures on hand.
add the following to crontab:
Code:
5 3 * * * /usr/bin/find /root/captures -type f -mtime +03 -exec rm -rf {} \;

What do we do now that we have files in our capture directory? Use the capital I(eye) flag for input, and lowercase l(ell) if you have a lot of calls on the system, to get over the default 20k message limit.

Code:
sngrep -I Date-within30MinsOfCall.pcap -l 50000

Use the filters in sngrep to pull up the call in question and boom! You have all the info you need and quickly! As soon as the call completes or during if your impatient...

That will do for pretty much all you could want to do. Another quick trick is that you can use the lowercase i flag to get just what you want at invocation.

Code:
#after the -i you can search for numbers, domain names, ips of endpoints or carriers etc
sngrep -I Date-within30MinsOfCall.pcap -i 5551234567

A note about sngrep: It leaks memory, and bad. If you leave sngrep running, it will consume all the ram on the system. Might take a while depending on call volume, even with the default 20k message limit, but do keep that in mind. It will also crash on some types of malformed sip which you will encounter as you capture, nearly for sure, so consider compiling from source if you find it crashing while opening captures. The beauty of this is that it's fast, no waiting for the call to get stored into db... the search time on homer is stupid, unless it’s really well resourced. If you love homer, keep on lovin’ it. You might just find this is quicker, though.


Enjoy, folks!

Ian
 

smn

Member
Jul 18, 2017
201
20
18
Getting the RTCP stuff working in Homer can be painful and that is the most useful part imo.

I usually just use tcpdump and watch stuff in real time. If it's too busy then I try filter it. Sngrep looks like a nice tool but I usually just use wireshark if I am capturing files.
 
Last edited:

Ian Oakes

Member
Oct 14, 2016
51
9
8
44
sngrep eliminates the need for gui, can do it from ssh easily enough. Tshark will spit out a sip ladder, but i find it much harder to read than sngrep - thus taking longer to get to the bottom of stuff for me.

Also all files are in one place, no need to transfer or download captures. Just open them in place!
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
i am using homer and like it really. especially the development regarding hep is getting better and better. so you simply need heplify and heplify server, to get the information into the homer-db. and then simply get call information into the database.
i am using asterisk, but am migrating to fusionpbx (not knowing yet how to get data into homer). on my job homer was the one and only tool we had to analyze calls. heplify is able to capture rtp, rtcp and sip data into files as well.
regards,
andre
 

phonesimon

Member
Apr 21, 2017
87
16
8
44
Homer is so buggy and frustrating, we went commercial and bought VoIP Monitor (voipmonitor.org) - and haven't looked back. Spend the money and enjoy your life more.
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
voipmonitor looks great, but my company wasn't able to spend that money. but that was not a problem, because we didn't encounter any issues.
could you tell me some of the bugs? just for information, don't want to convince you, but if i know the bugs, i may remember when i get hit by them. ;)
regards,
andre
 

phonesimon

Member
Apr 21, 2017
87
16
8
44
voipmonitor looks great, but my company wasn't able to spend that money. but that was not a problem, because we didn't encounter any issues.
could you tell me some of the bugs? just for information, don't want to convince you, but if i know the bugs, i may remember when i get hit by them. ;)
regards,
andre

Many UDP SIP packets were truncated, cutting off important information in INVITE packets. We worked with the homer team to get this solved but it seems their capture agent struggled with fragmented packets no matter what settings we used.

We run video in our network and homer failed to produce any stats on video; moreover, the rtcp and MOS stats (for audio) were incorrect on calls that included video streams.

I like HEP and sngrep and related tools but homer itself just wasn't working out.
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
ok, good to know. yes, i can confirm that we had issues with frgamented packets, but with udp i thought it would be our issue (wireshark has issues as well). but when we switched to heplify from captagent this issues were gone.
but i have no experience regarding video.
 

smn

Member
Jul 18, 2017
201
20
18
I think Homer is only half baked and seems to be largely abandoned. They will apply fixes if you submit pull requests. They seem to be concentrating on their commerical software. Can't really blame them.
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
i can't support your statement at all. sure, they have to keep the project alive by having their commercial product in focus, but bugs have usually been fixed quite fast.
 

smn

Member
Jul 18, 2017
201
20
18
i can't support your statement at all. sure, they have to keep the project alive by having their commercial product in focus, but bugs have usually been fixed quite fast.

It rarely just works without lots of tweaking and fiddling no matter how carefully you follow what documentation there is. Especially the RTCP stuff. If you are happy with it then by all means keep using it.
 

Ian Oakes

Member
Oct 14, 2016
51
9
8
44
I kinda agree with everyone, which is why I found an alternate workflow.

I know some people who think it is the greatest product ever, but in the end its resource intensive, doesn't capture tcp traffic correctly or at all, I have never had the RTCP stuff working correctly for both legs of the call blah blah blah. There is really nothing that justifies waiting some period of time to trace the call, either. If I am on with a customer and they are making test calls, neither they nor I want to wait even if it it only 30 seconds or so. I can go on, and in the end I will just reinforce why I decided not to continue using it in the first place.

The initial intent was less to say that homer is bad, more that for a variety of reasons I can do better.
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
I The initial intent was less to say that homer is bad, more that for a variety of reasons I can do better.

what do you choose instead of homer? i am always open to new things and i am quite interested. i was not able yet to find a better tool at that price (from nothing to some bucks for developing something for our needs).
for realtime i agree totally to use other tools (sngrep / tcpdump / wireshark / ...). but for customer complaints i always must be able to analyze the callflow of calls in the past.
 

DigitalDaz

Administrator
Staff member
Sep 29, 2016
3,070
577
113
what do you choose instead of homer? i am always open to new things and i am quite interested. i was not able yet to find a better tool at that price (from nothing to some bucks for developing something for our needs).
for realtime i agree totally to use other tools (sngrep / tcpdump / wireshark / ...). but for customer complaints i always must be able to analyze the callflow of calls in the past.

Just read the post from the beginning. Sngrep will allow you to chunk it up and then you can keep them for as long as you need.

I really like Homer but I just cannot get it to work correctly. It seems horrible when using larger packet sizes that you need TCP for.

When you then consider that with Homer every packet is shipped to a remote server, ie you effectively double the SIP traffic, you can start to see the appeal of the method used here.
 

smn

Member
Jul 18, 2017
201
20
18
I have been using sngrep quite a bit the last few days. Really like it. Gonna be my new best friend for troubleshooting SIP from now on.
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
I really like Homer but I just cannot get it to work correctly. It seems horrible when using larger packet sizes that you need TCP for.
we had similar issues and switched to heplify - it works for larger frames even with udp! and besides this it eliminates the need for complex captureagent.
 

brickrat

New Member
Sep 3, 2018
12
0
1
43
Hey All!


So as the title says, I am not real fond of homer. I suppose I could have spent more time with it and worked out all my grief, but ultimately its one or two more services and ips i have to devote to it. I can get what I want in a much simpler, and for me, faster way.

So, the trick here is using sngrep. If you haven't gotten acquainted, I would suggest doing so. It comes standard as part of the fusionpbx install now, which is cool!

Sngrep will show you a list of calls taken while its running, but like wireshark, tshark and homer, it will also read packet captures and show you the same thing. As i mentioned this can be used in near real time as sngrep can open running capture files, and give you calls up to the moment you opened it.

First, I am assuming sngrep is already installed, but if this is an older instance, follow the instructions here for package-based distros:
https://github.com/irontec/sngrep/wiki/Installing-Binaries

Alrighty, we are all installed, you are also going to need to install tcpdump and ngrep for fun, too.

Now, with that finished, you will want to run the following script:
Code:
#Full capture with media/rtp
#tcpdump -nq -s 0 -i eth0 -G1800 -w Sip_w_RTP_%Y-%m-%d_%H:%M:%S.pcap port 5060 or port 5080 or udp portrange 16384-32767 &

#Sip capture on 5060 and 5080
tcpdump -nq -s 0 -i eth0 -G1800 -w /root/captures/Sip_Only_%Y-%m-%d_%H:%M:%S.pcap port 5060 or port 5080 &

The two lines in there are for captures with media and without. Sometimes (rarely) you need to diagnose audio issues, so the first tcpdump line does that. You may have to modify the device, the ports, and the rtp range depending on whether your carrier sends on non-standard ports. I now use bash sngrep-capture.sh and it runs in the background. the -G flag specifies the interval to save files in this case its in 30 minute (1800 second) intervals to keep files manageable.

It will create the captures in the directory you specify, so watch the file path. You will want to leave this running all the time, so you will need to delete the files, which will eat up tons of space if you’re not careful. 3 days is a good range and I only ever have under a 100 gig of captures on hand.
add the following to crontab:
Code:
5 3 * * * /usr/bin/find /root/captures -type f -mtime +03 -exec rm -rf {} \;

What do we do now that we have files in our capture directory? Use the capital I(eye) flag for input, and lowercase l(ell) if you have a lot of calls on the system, to get over the default 20k message limit.

Code:
sngrep -I Date-within30MinsOfCall.pcap -l 50000

Use the filters in sngrep to pull up the call in question and boom! You have all the info you need and quickly! As soon as the call completes or during if your impatient...

That will do for pretty much all you could want to do. Another quick trick is that you can use the lowercase i flag to get just what you want at invocation.

Code:
#after the -i you can search for numbers, domain names, ips of endpoints or carriers etc
sngrep -I Date-within30MinsOfCall.pcap -i 5551234567

A note about sngrep: It leaks memory, and bad. If you leave sngrep running, it will consume all the ram on the system. Might take a while depending on call volume, even with the default 20k message limit, but do keep that in mind. It will also crash on some types of malformed sip which you will encounter as you capture, nearly for sure, so consider compiling from source if you find it crashing while opening captures. The beauty of this is that it's fast, no waiting for the call to get stored into db... the search time on homer is stupid, unless it’s really well resourced. If you love homer, keep on lovin’ it. You might just find this is quicker, though.


Enjoy, folks!

Ian
Hi

Thank you for this

I am new to all this sngrep and tcpdump etc. can i run this on seperate server to capture data on my fusion box?
 

astrakid

New Member
Apr 25, 2018
13
0
1
46
Hi

Thank you for this

I am new to all this sngrep and tcpdump etc. can i run this on seperate server to capture data on my fusion box?
you'd need to forward the traffic to another server (mirroring) and use the tools there. but that means more network traffic. you have to devide if it worth the effort.
we had to go that way because we were not able to install tools on an virtual appliance of avaya, so we did exactly that (but we used heplify to store pcaps and create HEPs for homer/sipcapture/kamailio).

edit: to sum up the previous discussion: i like sngrep as well and use it for realtime monitoring. it is really good. but homer is very helpful to search for calls you don't have an exact timestamp or look for events etc.

regards,
andre
 

DigitalDaz

Administrator
Staff member
Sep 29, 2016
3,070
577
113
Homer also take a hell of a lot of resources though. I have had nothing but trouble with it.
 

brickrat

New Member
Sep 3, 2018
12
0
1
43
I was looking at homer. Very slow and eats resources.

My setup is as follows
Endpoint —> fusionpbx —> astpp —> provider

So i am looking for best way and for now easiest way to capture sip packets
 
Status
Not open for further replies.