Re: [Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-08-03 Thread Fernando Testa
Hi, a bit late on answering some of the questions, but, here we go.On
Aculab, all codecs were G711u. The same codec we have on FS:

freeswi...@conference show channels
API CALL [show(channels)] output:
uuid,direction,created,created_epoch,name,state,cid_name,cid_num,ip_addr,dest,application,application_data,dialplan,context,read_codec,read_rate,write_codec,write_rate,secure
c9cafeb8-803a-11de-8ceb-cb8648fd1ccf,inbound,2009-08-03
11:34:44,1249310084,sofia/internal/1...@192.168.0.40,CS_EXECUTE,Teste
Testa,1000,192.168.0.165,3200,conference,3200-192.168.0...@ultrawideband
,XML,default,L16,8000,PCMU,8000,
cc5edb90-803a-11de-8ceb-cb8648fd1ccf,inbound,2009-08-03
11:34:49,1249310089,sofia/internal/1...@192.168.0.40
,CS_EXECUTE,F.G.Testa,1000,192.168.0.249,3200,conference,3200-192.168.0...@ultrawideband
,XML,default,L16,8000,PCMU,8000,

2 total.

freeswi...@conference conference list
API CALL [conference(list)] output:
Conference 3200-192.168.0.40 (2 members)
2;sofia/internal/1...@192.168.0.40
;cc5edb90-803a-11de-8ceb-cb8648fd1ccf;F.G.Testa;1000;hear|speak;0;0;300
1;sofia/internal/1...@192.168.0.40;c9cafeb8-803a-11de-8ceb-cb8648fd1ccf;Teste
Testa;1000;hear|speak|talking|floor;0;0;300

I think this answers some questions from Michael.
A packet dump I don't have right now.

Fernando G. Testa


On Thu, Jul 30, 2009 at 11:13 AM, Steve Underwood ste...@coppice.orgwrote:

 David Knell wrote:
  On Thu, 2009-07-30 at 09:21 +0800, Steve Underwood wrote:
 
 
  High quality conferencing is a difficult task, and still a research
  topic. No two conferencing systems perform alike. The interesting thing
  about this and other reports is that the conferencing in Freeswitch is
  not very clever right now, yet people are already saying it beats
  various other offerings, including long time commercial offerings.
 
 
  It may well be that a simplistic implementation (noise gate, add them
  all up) is all that's required for dealing with small groups or, more
  generally, groups of any size which have a small number of active
  speakers at any one time: it's predictable and unlikely to introduce
  unpleasant side effects.
 
 This is one of those situations where when you've experienced something
 better you make that your baseline for acceptability. I would consider a
 noise gate horribly crude, and VAD as the minimum for acceptable
 performance. If you've only used a noise gate you get used to it. If
 you're not sufficiently versed in the art you may well think nothing
 better is even possible.

 The fact that even the simple scheme, with noise gating, in Freeswitch
 is getting high praise, is pretty damning of mature commercial products.

 Steve


 ___
 FreeSWITCH-users mailing list
 FreeSWITCH-users@lists.freeswitch.org
 http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
 UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
 http://www.freeswitch.org




-- 
Fernando Gregianin Testa
Voice Technology Ltda
+55 11 35882166
___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


Re: [Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-07-30 Thread Steve Underwood
David Knell wrote:
 On Thu, 2009-07-30 at 09:21 +0800, Steve Underwood wrote:
   
  
 High quality conferencing is a difficult task, and still a research 
 topic. No two conferencing systems perform alike. The interesting thing 
 about this and other reports is that the conferencing in Freeswitch is 
 not very clever right now, yet people are already saying it beats 
 various other offerings, including long time commercial offerings.
 

 It may well be that a simplistic implementation (noise gate, add them
 all up) is all that's required for dealing with small groups or, more
 generally, groups of any size which have a small number of active
 speakers at any one time: it's predictable and unlikely to introduce
 unpleasant side effects.  
   
This is one of those situations where when you've experienced something 
better you make that your baseline for acceptability. I would consider a 
noise gate horribly crude, and VAD as the minimum for acceptable 
performance. If you've only used a noise gate you get used to it. If 
you're not sufficiently versed in the art you may well think nothing 
better is even possible.

The fact that even the simple scheme, with noise gating, in Freeswitch 
is getting high praise, is pretty damning of mature commercial products.

Steve


___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


[Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-07-29 Thread Fernando Testa
Hello freeswitchers!
I thought you're be pleased to know: FS beated Prosody S on subjective test
on lay users.
After 3 users testing the two conferences and reporting far better quality
on FS, we decided to make a more rigorous test.
Then we made a blind test by asking 4 lay users to dial two urls for 10
minutes from their x-lite and report the results. These users are non-techie
and never heard about FS at all.
The first url was Aculab Prosody S test app, then, a second call to FS on
default conf (extension 3000).
All users preferred FS. Some users were almost enthusiastic after entering
FS conf. Some even started to blame ProsodyS after hearing FS. The average
report told 50% more satisfaction on FS. Questions on report were simple
ones, with results from 1(worse) to 5(best) :
1-Overall opinion(1-crap; 5-awesome);
2-Delay (1-more delay, 5-no delay at all);
3-Audio quality (1-noisy, choppy, etc; 5-cristal clear);

Machines used:
Prosody S: Windows Vista Business, Intel Core 2 Duo E7400 @ 2.8 GHz, 4Gb de
RAM
FreeSwitch: CentOS 5.4, Intel(R) Pentium(R) 4 CPU 2.40GHz, 1Gb RAM

Congratulations! You made a great product!
-- 
Fernando Gregianin Testa
Voice Technology Ltda
+55 11 35882166
___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


Re: [Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-07-29 Thread Michael Collins
This is great news. Before we start sending out commercials that say 4 out
of 5 Brazilians prefer FreeSWITCH to Prosody I would very much like know
more about the tests:

What kind of computers were the end users working?
What codecs were being used by the x-lite clients when connecting to FS?
What codecs were being used by the x-lite clients when connecting to
Prosody?
Were there any other factors that could possibly have affected the call
quality?

The reason I ask all of these is that I would love to put this account into
the testimonials page on the wiki and make it available to other techie guys
who are trying to make their business cases for using FreeSWITCH (or FOSS in
general) to the corporate executives.

Thanks for the great report!
-Michael

On Wed, Jul 29, 2009 at 7:53 AM, Fernando Testa 
te...@voicetechnology.com.br wrote:

 Hello freeswitchers!
 I thought you're be pleased to know: FS beated Prosody S on subjective test
 on lay users.
 After 3 users testing the two conferences and reporting far better quality
 on FS, we decided to make a more rigorous test.
 Then we made a blind test by asking 4 lay users to dial two urls for 10
 minutes from their x-lite and report the results. These users are non-techie
 and never heard about FS at all.
 The first url was Aculab Prosody S test app, then, a second call to FS on
 default conf (extension 3000).
 All users preferred FS. Some users were almost enthusiastic after entering
 FS conf. Some even started to blame ProsodyS after hearing FS. The average
 report told 50% more satisfaction on FS. Questions on report were simple
 ones, with results from 1(worse) to 5(best) :
 1-Overall opinion(1-crap; 5-awesome);
 2-Delay (1-more delay, 5-no delay at all);
 3-Audio quality (1-noisy, choppy, etc; 5-cristal clear);

 Machines used:
 Prosody S: Windows Vista Business, Intel Core 2 Duo E7400 @ 2.8 GHz, 4Gb de
 RAM
 FreeSwitch: CentOS 5.4, Intel(R) Pentium(R) 4 CPU 2.40GHz, 1Gb RAM

 Congratulations! You made a great product!
 --
 Fernando Gregianin Testa
 Voice Technology Ltda
 +55 11 35882166


 ___
 FreeSWITCH-users mailing list
 FreeSWITCH-users@lists.freeswitch.org
 http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
 UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
 http://www.freeswitch.org


___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


Re: [Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-07-29 Thread David Knell
Hi Fernando,

Greetings from Rio..!

It'd be interesting to understand more about these results - roughly
speaking, two conferencing systems using the same codecs, etc., ought to
perform pretty much identically, particularly with just a few callers.

I'd be interested to see a network packet dump of a conference on each
of the machines, if you're able to make one available.

Cheers --

Dave


 Hello freeswitchers!
 
 
 I thought you're be pleased to know: FS beated Prosody S on subjective
 test on lay users. 
 After 3 users testing the two conferences and reporting far better
 quality on FS, we decided to make a more rigorous test.
 Then we made a blind test by asking 4 lay users to dial two urls for
 10 minutes from their x-lite and report the results. These users are
 non-techie and never heard about FS at all.
 The first url was Aculab Prosody S test app, then, a second call to FS
 on default conf (extension 3000).
 All users preferred FS. Some users were almost enthusiastic after
 entering FS conf. Some even started to blame ProsodyS after hearing
 FS. The average report told 50% more satisfaction on FS. Questions on
 report were simple ones, with results from 1(worse) to 5(best) :
 1-Overall opinion(1-crap; 5-awesome);
 2-Delay (1-more delay, 5-no delay at all);
 3-Audio quality (1-noisy, choppy, etc; 5-cristal clear);
 
 
 Machines used:
 Prosody S: Windows Vista Business, Intel Core 2 Duo E7400 @ 2.8
 GHz, 4Gb de RAM
 FreeSwitch: CentOS 5.4, Intel(R) Pentium(R) 4 CPU 2.40GHz, 1Gb RAM
 
 
 Congratulations! You made a great product!
 -- 
 Fernando Gregianin Testa
 Voice Technology Ltda
 +55 11 35882166
 
 
 ___
 FreeSWITCH-users mailing list
 FreeSWITCH-users@lists.freeswitch.org
 http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
 UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
 http://www.freeswitch.org
-- 
David Knell, Director, 3C Limited
T: +44 20 3298 2000
E: d...@3c.co.uk
W: http://www.3c.co.uk


___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org


Re: [Freeswitch-users] FS beats Aculab Prosody S on subjective test on lay users for conference quality

2009-07-29 Thread Steve Underwood
David Knell wrote:
 Hi Fernando,

 Greetings from Rio..!

 It'd be interesting to understand more about these results - roughly
 speaking, two conferencing systems using the same codecs, etc., ought to
 perform pretty much identically, particularly with just a few callers.

 I'd be interested to see a network packet dump of a conference on each
 of the machines, if you're able to make one available.
   
High quality conferencing is a difficult task, and still a research 
topic. No two conferencing systems perform alike. The interesting thing 
about this and other reports is that the conferencing in Freeswitch is 
not very clever right now, yet people are already saying it beats 
various other offerings, including long time commercial offerings.

Steve


___
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org