Thanks Rod, Its really helpful contribution.
@Nextone: I don't want to say much about it, but simply I am not happy with it, have you heard someone satisfied with NX who also owns it? Kind Regards Saeed Ahmed Tariq -----Original Message----- From: freeswitch-users-boun...@lists.freeswitch.org [mailto:freeswitch-users-boun...@lists.freeswitch.org] On Behalf Of rod Sent: Monday, February 02, 2009 3:33 PM To: freeswitch-users@lists.freeswitch.org Subject: Re: [Freeswitch-users] Strange Performance when using as SBC Hi Saeed, Here is a first draft of what I did to install FS on my server. Configuration are not present, they'll be in a next release :p http://wiki.freeswitch.org/wiki/SBC_Setup My aim is to setup FS as a SBC, I hope this page could be a great startup point for others. I will update regularly based on what I did. Saeed, why are you replacing your Nextone, it's said to be one of the best commercial SBC on the market. regards. Saeed Ahmed wrote: > Thanks rod for a quick answer, > > FS is installed on Ubuntu Server. > > I am planning to replace Nextone SBC with FS, Later I'll also use openZAP to > communicate with TDM but this all depends how much calls it can take, or > maybe we can also do something in clustering environment ( I am not sure > about it). But thanks again and any further help will be highly appreciated. > > > Kind Regards > Saeed Ahmed Tariq > > > > -----Original Message----- > From: freeswitch-users-boun...@lists.freeswitch.org > [mailto:freeswitch-users-boun...@lists.freeswitch.org] On Behalf Of rod > Sent: Monday, February 02, 2009 1:53 PM > To: freeswitch-users@lists.freeswitch.org > Subject: Re: [Freeswitch-users] Strange Performance when using as SBC > > Hi Saeed, > > I just created an account to share my setup on the wiki. I will detail > all the steps for a clean install of a debian64 lenny with FS used as a > SBC (next step is to try the new LCR module :) )and what I'm doing do > stress the server. > > I wrote nothing at this time so please be patient, I'm waiting for my > new hardware so that I will detail as much as possible what I'll do. > > For beginning I suggest you reading the start page on the wiki, > especially these pages: > -http://wiki.freeswitch.org/wiki/Getting_Started_Guide > -http://wiki.freeswitch.org/wiki/Dialplan_XML > > maybe you could tell more about the linux distribution you're using so > that I can give you some pointers for sipp... > > regards. > rod. > > > Saeed Ahmed wrote: > >> Hi Rod, >> >> Could you please share how you configured Sipp & FS to create a test >> environment? Especially the dial plan, sofia settings etc..., actually I >> > am > >> a newbie. I want to test it on a single FS machine. >> >> Kind Regards >> Saeed >> -----Original Message----- >> From: freeswitch-users-boun...@lists.freeswitch.org >> [mailto:freeswitch-users-boun...@lists.freeswitch.org] On Behalf Of rod >> Sent: Monday, February 02, 2009 11:00 AM >> To: freeswitch-users@lists.freeswitch.org >> Subject: Re: [Freeswitch-users] Strange Performance when using as SBC >> >> Hi Ken, Jay, >> >> thanks for pointing to proxy media, I will test. >> >> Ken, you are right, I was brain damaged (a stupid mistake) when setting >> INFO cause this kind of level could be very verbose. I'm switching to >> CRIT or ERR. >> >> Thanks guys, >> rod. >> >> thanks for >> >> Ken Rice wrote: >> >> >>> If you don't have to transcode, using proxy media mode will still save >>> > you > >>> some CPU time. This is 1/2 way between bypass media and the default media >>> interactive mode. The other draw back to this mode is if you are using FS >>> >>> >> to >> >> >>> clean up RTP and DTMF you loose those functions but they are not needed >>> > in > >>> most use cases. >>> >>> As far as the log level goes, I found that once I had things stable >>> >>> >> setting >> >> >>> the loglevel to helped a good deal... Info is probably a bit too high of >>> > a > >>> loglevel I would probably go for CRIT or ERR (2 or 1 respectively) if you >>> insist on leaving logging turned on... On a busy system these can and >>> > will > >>> generate a good deal of activity (and disk IO if using mod_logfile) >>> >>> Ken >>> >>> >>> >>> >>> >>>> From: rod <kawa...@laposte.net> >>>> Reply-To: <freeswitch-users@lists.freeswitch.org> >>>> Date: Mon, 02 Feb 2009 11:36:35 +0400 >>>> To: <freeswitch-users@lists.freeswitch.org> >>>> Subject: Re: [Freeswitch-users] Strange Performance when using as SBC >>>> >>>> Hi Ken, >>>> >>>> 1) I'd like to use FS to hide topology, so bypass media is not possible >>>> 2) done >>>> 3) done >>>> 4) not used >>>> 5) i'm using this ins switch.xml -> <param name="loglevel" >>>> value="info"/>, if you think an other log level is more suitable. >>>> >>>> Regarding logging, I can see in console and in the freeswitch.log that >>>> there is still a lot of NOTICE logging, see below: >>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960 >>>> switch_core_session_thread() Session 8721 >>>> (sofia/internal/s...@10.10.10.1:5060) Ended >>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962 >>>> switch_core_session_thread() Close Channel >>>> sofia/internal/s...@10.10.10.1:5060 [CS_HANGUP] >>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960 >>>> switch_core_session_thread() Session 8722 >>>> (sofia/external/9...@10.10.20.100) Ended >>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962 >>>> switch_core_session_thread() Close Channel >>>> sofia/external/9...@10.10.20.100 [CS_HANGUP] >>>> 2009-02-02 08:33:56 [NOTICE] sofia.c:3164 sofia_handle_sip_i_state() >>>> Channel [sofia/external/9...@10.10.20.100] has been answered >>>> 2009-02-02 08:33:56 [WARNING] mod_sofia.c:740 sofia_read_frame() >>>> Changing codec ptime to 30. I bet you have a linksys/sipura =D >>>> >>>> Do you have any idea where I can switch off this kind of logging. I >>>> thought it should be in /dialplan/internal.xml, but I see that in >>>> internal.xml -> <param name="debug" value="0"/> >>>> >>>> thanks a lot for your suggestion. >>>> >>>> regards, >>>> rod >>>> >>>> Ken Rice wrote: >>>> >>>> >>>> >>>>> Dont forget there are several things you can do to increase >>>>> >>>>> >> performance... >> >> >>>>> 1) where possible use bypass media or media proxy modes >>>>> 2) mount freeswitch/db as a ram drive (if you are using voicemail with >>>>> the internal FS DBs you'll need a way to make this persistant across >>>>> reboots) >>>>> 3) see the wiki for setting reasonable ulimits >>>>> 4) (this is my oppinion others may vary) dont use mod_cdr_csv >>>>> 5) turn off (or reduce logging) in switch.conf.xml >>>>> >>>>> all of these thing can greatly improve performance. >>>>> >>>>> On Mon, Feb 2, 2009 at 1:04 AM, rod <kawa...@laposte.net >>>>> <mailto:kawa...@laposte.net>> wrote: >>>>> >>>>> Thanks Anthony, >>>>> >>>>> the setup is like this: >>>>> >>>>> sipp server ---- FS 1 ---- FS2 >>>>> >>>>> FS1 is the AMD CPU that has only one extension in dialplan that >>>>> bridges >>>>> 9999 to FS2. 9999 is the first extension in FS2 dialplan that >>>>> plays moh, >>>>> FS2 has no CPU pbm. >>>>> >>>>> FS1 is maxing out at 60 bridged calls without your option -hp. >>>>> >>>>> Using -hp, I'm now able to bridge 200 concurrent calls (a great >>>>> improvement) and the system is still reactive. CPU load is high >>>>> but not >>>>> 100% and as the system responds well, I think that doesn't matter. >>>>> >>>>> >> The >> >> >>>>> 2GB of memory are completely consumed (top command shows 700MB for >>>>> >>>>> >> FS >> >> >>>>> process). >>>>> >>>>> I understand that FS1 server is not the best hardware platform, >>>>> and I'm >>>>> waiting for new 4 cores server for testing. >>>>> I will update those numbers when testing with the new hardware. >>>>> >>>>> regards, >>>>> rod. >>>>> >>>>> Anthony Minessale wrote: >>>>> >>>>> >>>>> >>>>>> Which of the 2 machines has the load issue? You said it was one box >>>>>> calling the other. >>>>>> >>>>>> You have 2 major things against you, single CPU and AMD, but you >>>>>> should at least be able to get in the vicinity of 800-1000 calls >>>>>> >>>>>> >>>>>> >>>>> on a >>>>> >>>>> >>>>> >>>>>> box like that. >>>>>> >>>>>> Are you calling the default 9999? It's not really an appropriate >>>>>> extension for load testing. >>>>>> On the terminating box you should set up a manual extension that is >>>>>> the first one in the dial plan >>>>>> to play a wav file from preferably a ram disk or /tmp >>>>>> >>>>>> If you do plan on using this in production accept nothing less >>>>>> >>>>>> >>>>>> >>>>> than a >>>>> >>>>> >>>>> >>>>>> multi-core intel machine with at least 4 cores, the more cores the >>>>>> better because that parallel processing is where FS gets it's >>>>>> >>>>>> >>>>>> >>>>> atvantage. >>>>> >>>>> >>>>> >>>>>> On Fri, Jan 30, 2009 at 5:56 AM, rod <kawa...@laposte.net >>>>>> >>>>>> >>>>>> >>>>> <mailto:kawa...@laposte.net> >>>>> >>>>> >>>>> >>>>>> <mailto:kawa...@laposte.net <mailto:kawa...@laposte.net>>> wrote: >>>>>> >>>>>> Dear list, >>>>>> >>>>>> I've been playing with freeswitch for some time (2 months) >>>>>> >>>>>> >>>>>> >>>>> and the >>>>> >>>>> >>>>> >>>>>> fact >>>>>> is that I'm very pleased with the functionnalities of this >>>>>> >>>>>> >>>>>> >>>>> software. >>>>> >>>>> >>>>> >>>>>> I'd like to use FS as a SBC handling media and I'm doing some >>>>>> tests with >>>>>> sipp to load the machine but I'm unable to bridge more than >>>>>> >>>>>> >>>>>> >>>>> 60 calls >>>>> >>>>> >>>>> >>>>>> without seeing the CPU being loaded at 100%. I'm sure >>>>>> >>>>>> >>>>>> >>>>> something is >>>>> >>>>> >>>>> >>>>>> going >>>>>> wrong with my setup but I'm unable to see what. >>>>>> >>>>>> The test machine has the following specs: >>>>>> Athlon XP 3500+ with 2GB of memory (I know this is not a >>>>>> >>>>>> >>>>>> >>>>> high end >>>>> >>>>> >>>>> >>>>>> machine :p) >>>>>> >>>>>> Freeswitch:/opt/freeswitch/log# cat /proc/cpuinfo >>>>>> processor : 0 >>>>>> vendor_id : AuthenticAMD >>>>>> cpu family : 15 >>>>>> model : 95 >>>>>> model name : AMD Athlon(tm) 64 Processor 3500+ >>>>>> stepping : 2 >>>>>> cpu MHz : 2199.973 >>>>>> cache size : 512 KB >>>>>> fpu : yes >>>>>> fpu_exception : yes >>>>>> cpuid level : 1 >>>>>> wp : yes >>>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic >>>>>> >>>>>> >>>>>> >>>>> sep mtrr pge >>>>> >>>>> >>>>> >>>>>> mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext >>>>>> fxsr_opt >>>>>> rdtscp lm 3dnowext 3dnow up rep_good pni cx16 lahf_lm svm >>>>>> >>>>>> >>>>>> >>>>> extapic >>>>> >>>>> >>>>> >>>>>> cr8_legacy >>>>>> bogomips : 4402.97 >>>>>> TLB size : 1024 4K pages >>>>>> clflush size : 64 >>>>>> cache_alignment : 64 >>>>>> address sizes : 40 bits physical, 48 bits virtual >>>>>> power management: ts fid vid ttp tm stc >>>>>> >>>>>> I installed FS on a fresh debian 64: >>>>>> Linux Freeswitch 2.6.26-1-amd64 #1 SMP Sat Jan 10 17:57:00 >>>>>> >>>>>> >>>>>> >>>>> UTC 2009 >>>>> >>>>> >>>>> >>>>>> x86_64 GNU/Linux >>>>>> >>>>>> I set the ulimit parameters like those on the website: >>>>>> freeswi...@internal> ... >>>>>> Freeswitch:/opt/free-svn/bin# ulimit -a >>>>>> core file size (blocks, -c) unlimited >>>>>> data seg size (kbytes, -d) unlimited >>>>>> scheduling priority (-e) 0 >>>>>> file size (blocks, -f) unlimited >>>>>> pending signals (-i) unlimited >>>>>> max locked memory (kbytes, -l) unlimited >>>>>> max memory size (kbytes, -m) unlimited >>>>>> open files (-n) 999999 >>>>>> pipe size (512 bytes, -p) 8 >>>>>> POSIX message queues (bytes, -q) unlimited >>>>>> real-time priority (-r) 0 >>>>>> stack size (kbytes, -s) 244 >>>>>> cpu time (seconds, -t) unlimited >>>>>> max user processes (-u) unlimited >>>>>> virtual memory (kbytes, -v) unlimited >>>>>> file locks (-x) unlimited >>>>>> >>>>>> >>>>>> My network setup is the following: >>>>>> >>>>>> SIPP machine (10.10.10.1/24)----------------vlan >>>>>> >>>>>> >>>>>> >>>>> <http://10.10.10.1/24%29----------------vlan> >>>>> >>>>> >>>>> >>>>>> <http://10.10.10.1/24%29----------------vlan> 55 >>>>>> ----------(10.10.10.254/24 <http://10.10.10.254/24> >>>>>> >>>>>> >>>>>> >>>>> <http://10.10.10.254/24>) FS >>>>> >>>>> >>>>> >>>>>> (10.10.20.254/24)-------------- >>>>>> >>>>>> >>>>>> >>>>> <http://10.10.20.254/24%29--------------> >>>>> >>>>> >>>>> >>>>>> <http://10.10.20.254/24%29--------------> vlan56 >>>>>> -------------------(10.10.20.100/24 <http://10.10.20.100/24> >>>>>> >>>>>> >>>>>> >>>>> <http://10.10.20.100/24>) >>>>> >>>>> >>>>> >>>>>> OTHER STOCK FS >>>>>> >>>>>> >>>>>> I launched sipp with: >>>>>> sipp -sn uac_pcap -s 9999 -r 10 -l 80 -d 60000 -mi >>>>>> >>>>>> >>>>>> >>>>> 10.10.10.1 -i >>>>> >>>>> >>>>> >>>>>> 10.10.10.1 -mp 25000 10.10.10.254:5060 >>>>>> >>>>>> >>>>>> >>>>> <http://10.10.10.254:5060> <http://10.10.10.254:5060> >>>>> >>>>> >>>>> >>>>>> The dialplan on FS is very simple: >>>>>> <?xml version="1.0" encoding="utf-8"?> >>>>>> <!-- http://wiki.freeswitch.org/wiki/Dialplan_XML --> >>>>>> <include> >>>>>> <context name="default"> >>>>>> >>>>>> <extension name="hold_music_bridge"> >>>>>> <condition field="destination_number" expression="^9999$"> >>>>>> <action application="answer"/> >>>>>> <action application="bridge" >>>>>> data="sofia/external/9...@10.10.20.100 >>>>>> >>>>>> >>>>>> >>>>> <mailto:9...@10.10.20.100> <mailto:9...@10.10.20.100 >>>>> <mailto:9...@10.10.20.100>>"/> >>>>> >>>>> >>>>> >>>>>> </condition> >>>>>> </extension> >>>>>> </context> >>>>>> >>>>>> </include> >>>>>> >>>>>> FreeSWITCH Version 1.0.trunk (11560M) Started. >>>>>> Crash Protection [Disabled] >>>>>> Max Sessions[1000] >>>>>> Session Rate[100] >>>>>> SQL [Enabled] >>>>>> >>>>>> >>>>>> The test is very simple: sipp dial 9999 that matches in my >>>>>> >>>>>> >>>>>> >>>>> FS dialplan >>>>> >>>>> >>>>> >>>>>> and this is bridged to an other FS machine playing music on >>>>>> >>>>>> >>>>>> >>>>> hold. >>>>> >>>>> >>>>> >>>>>> When I launch "top" I see after 30 to 40 s that FS consumes all >>>>>> the CPU >>>>>> ressources (with a mean of 50-60 % before), with 80 calls. >>>>>> When I set 70 calls, I have to wait 70-80 s before seeing >>>>>> >>>>>> >>>>>> >>>>> the same >>>>> >>>>> >>>>> >>>>>> issue. >>>>>> >>>>>> Presence is set to false on the 2 profile. >>>>>> >>>>>> I have the same issue with FS 1.0.2 that' s why I tried FS >>>>>> >>>>>> >>>>>> >>>>> 11560. >>>>> >>>>> >>>>> >>>>>> When I use the FS machine as a router to test the packet per >>>>>> >>>>>> >>>>>> >>>>> second >>>>> >>>>> >>>>> >>>>>> performance, I'm reaching 100Mbps with 8000pps in each >>>>>> >>>>>> >>>>>> >>>>> direction (from >>>>> >>>>> >>>>> >>>>>> vlan 55 to vlan56) with less than 12% CPU. So that I don't think >>>>>> there's >>>>>> an issue with the network. >>>>>> >>>>>> Here is an "mpstat -P ALL 1" to show you what's happening >>>>>> >>>>>> >>>>>> >>>>> suddenly >>>>> >>>>> >>>>> >>>>>> with >>>>>> 70 bridge calls: >>>>>> 12:31:26 CPU %user %nice %sys %iowait %irq %soft >>>>>> %steal %idle intr/s >>>>>> 12:31:27 all 3,00 0,00 3,00 0,00 1,00 4,00 >>>>>> 0,00 89,00 6241,00 >>>>>> 12:31:27 0 3,00 0,00 3,00 0,00 1,00 4,00 >>>>>> 0,00 89,00 6241,00 >>>>>> >>>>>> 12:31:27 CPU %user %nice %sys %iowait %irq %soft >>>>>> %steal %idle intr/s >>>>>> 12:31:28 all 14,14 0,00 56,57 0,00 2,02 5,05 >>>>>> 0,00 22,22 6035,35 >>>>>> 12:31:28 0 14,14 0,00 56,57 0,00 2,02 5,05 >>>>>> 0,00 22,22 6035,35 >>>>>> >>>>>> 12:31:28 CPU %user %nice %sys %iowait %irq %soft >>>>>> %steal %idle intr/s >>>>>> 12:31:29 all 24,75 0,00 67,33 0,00 0,99 6,93 >>>>>> 0,00 0,00 5483,17 >>>>>> 12:31:29 0 24,75 0,00 67,33 0,00 0,99 6,93 >>>>>> 0,00 0,00 5483,17 >>>>>> >>>>>> >>>>>> The CPU is going from 89% idle to 0% in less than 2 seconds. >>>>>> >>>>>> I know that I don't have to expect too much from this kind of >>>>>> hardware, >>>>>> but it seems strange that the CPU power vanished so suddenly. >>>>>> >>>>>> Thanks a lot for the guys that have read this long mail :p >>>>>> >>>>>> kind regards, >>>>>> rod >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Freeswitch-users mailing list >>>>>> Freeswitch-users@lists.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:Freeswitch-users@lists.freeswitch.org> >>>>> >>>>> >>>>> >>>>>> <mailto:Freeswitch-users@lists.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:Freeswitch-users@lists.freeswitch.org>> >>>>> >>>>> >>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>>>>> >>>>>> >>>>>> >>>>>> >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> >> >>>>> >>>>> >>>>> >>>>>> http://www.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Anthony Minessale II >>>>>> >>>>>> FreeSWITCH http://www.freeswitch.org/ >>>>>> ClueCon http://www.cluecon.com/ >>>>>> >>>>>> AIM: anthm >>>>>> MSN:anthony_miness...@hotmail.com >>>>>> >>>>>> >>>>>> >>>>> <mailto:msn%3aanthony_miness...@hotmail.com> >>>>> >>>>> >>>>> >>>>>> <mailto:msn%3aanthony_miness...@hotmail.com >>>>>> >>>>>> >>>>>> >>>>> <mailto:msn%253aanthony_miness...@hotmail.com>> >>>>> >>>>> >>>>> >>>>>> GTALK/JABBER/PAYPAL:anthony.miness...@gmail.com >>>>>> >>>>>> >>>>>> >>>>> <mailto:paypal%3aanthony.miness...@gmail.com> >>>>> >>>>> >>>>> >>>>>> <mailto:paypal%3aanthony.miness...@gmail.com >>>>>> >>>>>> >>>>>> >>>>> <mailto:paypal%253aanthony.miness...@gmail.com>> >>>>> >>>>> >>>>> >>>>>> IRC: irc.freenode.net <http://irc.freenode.net> >>>>>> >>>>>> >>>>>> >>>>> <http://irc.freenode.net> #freeswitch >>>>> >>>>> >>>>> >>>>>> FreeSWITCH Developer Conference >>>>>> sip:8...@conference.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:sip%3a...@conference.freeswitch.org> >>>>> >>>>> >>>>> >>>>>> <mailto:sip%3a...@conference.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:sip%253a...@conference.freeswitch.org>> >>>>> >>>>> >>>>> >>>>>> iax:gu...@conference.freeswitch.org/888 >>>>>> >>>>>> >>>>>> >>>>> <http://iax:gu...@conference.freeswitch.org/888> >>>>> >>>>> >>>>> >>>>>> <http://iax:gu...@conference.freeswitch.org/888> >>>>>> googletalk:conf+...@conference.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:googletalk%3aconf%2b...@conference.freeswitch.org> >>>>> >>>>> >>>>> >>>>>> <mailto:googletalk%3aconf%2b...@conference.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:googletalk%253aconf%252b...@conference.freeswitch.org>> >>>>> >>>>> >>>>> >>>>>> pstn:213-799-1400 >>>>>> >>>>>> >>>>>> >>>>>> >> ------------------------------------------------------------------------ >> >> >>>>> >>>>> >>>>> >>>>>> _______________________________________________ >>>>>> Freeswitch-users mailing list >>>>>> Freeswitch-users@lists.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>> <mailto:Freeswitch-users@lists.freeswitch.org> >>>>> >>>>> >>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>>>>> >>>>>> >>>>>> >>>>>> >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> >> >>>>> >>>>> >>>>> >>>>>> http://www.freeswitch.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Freeswitch-users mailing list >>>>> Freeswitch-users@lists.freeswitch.org >>>>> <mailto:Freeswitch-users@lists.freeswitch.org> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>>>> >>>>> >>>>> >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> >> >>>>> http://www.freeswitch.org >>>>> >>>>> >>>>> >>>>> > ------------------------------------------------------------------------ > >>>>> _______________________________________________ >>>>> Freeswitch-users mailing list >>>>> Freeswitch-users@lists.freeswitch.org >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>>>> >>>>> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > >>>>> http://www.freeswitch.org >>>>> >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Freeswitch-users mailing list >>>> Freeswitch-users@lists.freeswitch.org >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >>>> http://www.freeswitch.org >>>> >>>> >>>> >>> _______________________________________________ >>> Freeswitch-users mailing list >>> Freeswitch-users@lists.freeswitch.org >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >>> http://www.freeswitch.org >>> >>> >>> >>> >>> >> _______________________________________________ >> Freeswitch-users mailing list >> Freeswitch-users@lists.freeswitch.org >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> http://www.freeswitch.org >> >> >> _______________________________________________ >> Freeswitch-users mailing list >> Freeswitch-users@lists.freeswitch.org >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> http://www.freeswitch.org >> >> >> >> > > _______________________________________________ > Freeswitch-users mailing list > Freeswitch-users@lists.freeswitch.org > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org > > > _______________________________________________ > Freeswitch-users mailing list > Freeswitch-users@lists.freeswitch.org > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org > > > _______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org _______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org