Dan, only the decoder threads are per export and ACKs are sent by the TCP
layer itself. I am pretty sure clients do send multiple requests on the
same socket (we have seen it exceed the default 512 after which we drop
requests) and we use multiple worker threads.
Regards, Malahal.
On Wed, Feb 14, 2018 at 7:02 PM, Daniel Gryniewicz <d...@redhat.com> wrote:
> How many clients are you using? Each client op can only (currently) be
> handled in a single thread, and client's won't send more ops until the
> current one is ack'd, so Ganesha can basically only parallelize on a
> per-client basis at the moment.
>
> I'm sure there are locking issues; so far we've mostly worked on
> correctness rather than performance. 2.6 has changed the threading model a
> fair amount, and 2.7 will have more improvements, but it's a slow process.
>
> Daniel
>
> On 02/13/2018 06:38 PM, Deepak Jagtap wrote:
>
>> Thanks Daniel!
>>
>> Yeah user-kernel context switching is definitely adding up latency, but I
>> wonder ifrpc or some locking overhead is also in the picture.
>>
>> With 70% read 30% random workload nfs ganesha CPU usage was close to 170%
>> while remaining 2 cores were pretty much unused (~18K IOPS, latency ~8ms)
>>
>> With 100% read 30% random nfs ganesha CPU usage ~250% ( ~50K IOPS,
>> latency ~2ms).
>>
>>
>> -Deepak
>>
>> ------------------------------------------------------------------------
>> *From:* Daniel Gryniewicz <d...@redhat.com>
>> *Sent:* Tuesday, February 13, 2018 6:15:47 AM
>> *To:* nfs-ganesha-devel@lists.sourceforge.net
>> *Subject:* Re: [Nfs-ganesha-devel] nfs ganesha vs nfs kernel performance
>> Also keep in mind that FSAL VFS can never, by it's very nature, beat
>> knfsd, since it has to do everything knfsd does, but also has userspace
>> <-> kernespace transitions. Ganesha's strength is exporting
>> userspace-based cluster filesystems.
>>
>> That said, we're always working to make Ganesha faster, and I'm sure
>> there's gains to be made, even in these circumstances.
>>
>> Daniel
>>
>>
>> On 02/12/2018 07:01 PM, Deepak Jagtap wrote:
>>
>>> Hey Guys,
>>>
>>>
>>> I ran few performance tests to compare nfs gansha and nfs kernel server
>>> and noticed significant difference.
>>>
>>>
>>> Please find my test result:
>>>
>>>
>>> SSD formated with EXT3 exported using nfs ganesha : ~18K IOPS Avg
>>> latency: ~8ms Throughput: ~60MBPS
>>>
>>> same directory exported using nfs kernel server: ~75K IOPS
>>> Avg latency: ~0.8ms Throughput: ~300MBPS
>>>
>>>
>>> nfs kernel and nfs ganesha both of them are configured with 128
>>> worker threads. nfs ganesha is configured with VFS FSAL.
>>>
>>>
>>> Am I missing something major in nfs ganesha config or this is expected
>>> behavior.
>>>
>>> Appreciate any inputs as how the performance can be improved for nfs
>>> ganesha.
>>>
>>>
>>>
>>> Please find following ganesha config file that I am using:
>>>
>>>
>>> NFS_Core_Param
>>> {
>>> Nb_Worker = 128 ;
>>> }
>>>
>>> EXPORT
>>> {
>>> # Export Id (mandatory, each EXPORT must have a unique Export_Id)
>>> Export_Id = 77;
>>> # Exported path (mandatory)
>>> Path = /host/test;
>>> Protocols = 3;
>>> # Pseudo Path (required for NFS v4)
>>> Pseudo = /host/test;
>>> # Required for access (default is None)
>>> # Could use CLIENT blocks instead
>>> Access_Type = RW;
>>> # Exporting FSAL
>>> FSAL {
>>> Name = VFS;
>>> }
>>> CLIENT
>>> {
>>> Clients = *;
>>> Squash = None;
>>> Access_Type = RW;
>>> }
>>> }
>>>
>>>
>>>
>>> Thanks & Regards,
>>>
>>> Deepak
>>>
>>>
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>
>>>
>>>
>>> _______________________________________________
>>> Nfs-ganesha-devel mailing list
>>> Nfs-ganesha-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>>>
>>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Nfs-ganesha-devel mailing list
>> Nfs-ganesha-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
