Re: [Gluster-devel] I missed a patch in earlier releases of 3.7.x which is breaking virt usecase

[Pranith Kumar Karampuri]

On Sat, Jul 30, 2016 at 7:05 AM, Kaushal Madappa  wrote:

> On 29 Jul 2016 23:16, "Pranith Kumar Karampuri" 
> wrote:
> >
> > Krutika RC'd that I missed a patch which broke virt usecase.
> http://review.gluster.org/15050 is posted for this bug. Please don't
> release without this one. I will be available until this patch is merged in
> the morning...
>
> If someone reviews with a +1 I'll merge.
>

It is a straight backport. I merged it.


> >
> > --
> > Pranith
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Proposing a framework to leverage existing Python unit test standards for our testing

[Jonathan Holloway]

- Original Message -

> From: "Amye Scavarda" 
> To: "Vijay Bellur" 
> Cc: "Kaushal M" , "Jonathan Holloway"
> , "Gluster Devel" 
> Sent: Sunday, July 24, 2016 7:53:06 PM
> Subject: Re: [Gluster-devel] Proposing a framework to leverage existing
> Python unit test standards for our testing

> On Wed, Jul 20, 2016 at 1:24 PM, Vijay Bellur < vbel...@redhat.com > wrote:

> > On 07/20/2016 08:43 AM, Kaushal M wrote:
> 

> > > On Wed, Jul 20, 2016 at 1:41 PM, Jonathan Holloway < jhollo...@redhat.com
> > > >
> > > wrote:
> > 
> 

> > > > Hi Gluster-Devel,
> > > 
> > 
> 

> > > > There's been some conversation about standard Python unit test formats
> > > > (PyUnit, PyTest, Nose) and potentially leveraging a tool I've been
> > > > working
> > > > on (called Glusto) that wraps those standards as well as covers the
> > > > fundamentals required of the DiSTAF framework. I'm reaching out to
> > > > propose
> > > > this to the Gluster-Devel Community for consideration.
> > > 
> > 
> 

> > > Finally! I'd been waiting for ever wondering we would start discussing
> > 
> 
> > > this in the community. Thanks for starting this Jonathan.
> > 
> 

> > > > Some of the primary features Glusto offers are:
> > > 
> > 
> 
> > > > - Reads and writes yaml, json, and ini config file formats (including
> > > > Ansible
> > > > host files).
> > > 
> > 
> 
> > > > - Provides SSH, RPyC, logging (w/ ANSI color support), configuration,
> > > > templating (via Jinja), and simple REST methods.
> > > 
> > 
> 
> > > > - Implements cartesian product combinations with standard PyUnit class
> > > > format
> > > > for the Gluster runs_on_volumes/runs_on_mounts/reuse-setup
> > > > requirements.
> > > 
> > 
> 
> > > > - Wraps the Python standard framework modules (PyUnit, PyTest, Nose) in
> > > > a
> > > > single command with a config file option.
> > > 
> > 
> 
> > > > - Tests can also be run from the CLI, IDLE, or unittest savvy tools
> > > > (e.g.,
> > > > Eclipse PyDev).
> > > 
> > 
> 
> > > > - Glusto methods can also be used from IDLE for troubleshooting during
> > > > development--as well as in scripts.
> > > 
> > 
> 
> > > > - Allows for leveraging existing unit test features such as skip
> > > > decorators,
> > > > pytest markers, etc.
> > > 
> > 
> 

> > > These all seem really good! I particularly like the idea of having the
> > 
> 
> > > ability to use standard python test frameworks.
> > 
> 

> > > Glusto is something that DiSTAF core would have become, just that it's
> > > now.
> > 
> 
> > > The work done to get test generation working (cartesian products),
> > 
> 
> > > also shows it's flexible as well.
> > 
> 

> > > Glusto+DiSTAF libs seems to me will be a good combination.
> > 
> 

> > Agree here.
> 

> > > > I know this was a brief and high-level intro to Glusto. This is just to
> > > > get
> > > > the topic started, and we can cover details in discussion.
> > > 
> > 
> 

> > > Having a demo of glusto would be nice. Even a recorded demo would be
> > 
> 
> > > good as well.
> > 
> 

> > +1. Can we schedule a demo of glusto over a hangout or bluejeans for the
> > community?
> 

> I would love to see a recorded demo for this. Jonathan, does this upcoming
> week work to review this, or should we put something on the calendars for
> next week?

I've posted several videos on YouTube at the following links. 
There are eight sections and then a combined full-length (really full) video. 
There is a little bit of Unit Test covered in "3. Using Glusto Overview", but 
the "8. Running Unit Tests" shows more depth (sample PyUnit format w/ some 
PyTest, Gluster runs-on and reuse-setup example, filtering test runs, etc.). 
If you're looking to skip around, you might start with "1. Intro, 2. Using 
Glusto Overview, and "8. Running Unit Tests"--then pick and choose from there. 

1. Intro https://www.youtube.com/watch?v=IlG6Dl3CoxQ (1:28) 
2. Installing https://www.youtube.com/watch?v=FPgWxFJT2TA (2:12) 
3. Using Glusto Overview https://www.youtube.com/watch?v=SQYhyxJUrBc (7:39) 
4. SSH https://www.youtube.com/watch?v=KBo6hGaiWpo (10:26) 
5. RPyC https://www.youtube.com/watch?v=Ibp3YhFtofA (10:28) 
6. Configs https://www.youtube.com/watch?v=DhUPqPOXMDY (8:44) 
7. Logging https://www.youtube.com/watch?v=yf_tNHk88Ok (6:39) 
8. Running Unit Tests https://www.youtube.com/watch?v=Cu_lZvEcn1w (25:44) 
Combined Sessions 1-8 https://www.youtube.com/watch?v=LtG2sjYAlFE (1:13:16) 

I used a couple of docker images for each of the demos, so I'll record 
something on docker next week as well. 
I didn't really tweak audio, so levels might hop around a bit. I opted to use 
all open source recording/editing tools for the first time (instead of the Mac 
*gasp*), so pardon my mess while I figure things out. :-) 

Cheers, 
Jonathan 

> - amye

> > Thank you for posting about glusto. I look forward to checking out its
> > capabilities.
> 

> > Regards,
> 

Re: [Gluster-devel] Duplicati UUID entries

[Atin Mukherjee]

On Friday 29 July 2016, ABHISHEK PALIWAL  wrote:

> Hi,
>
> After a long time I am posting one more issue here.
>
> We have two board and glusterfs in sync on both of them, and our test case
> to restart one board continuously but in this TestCase we are getting
> duplicate entries of UUID in "gluster peer status" command and it is very
> rarely seen.
>
>
> So, I just want to know the possible reason behind this problem.
>
> There is no possibility of deletion of glusterd.info file and I think if
> this is not present then only gluster will generate new UUID for the peer.
>

That's right but based on our earlier email exchanges for the other issues
you reported I believe when a board is restarted the content of
/var/lib/glusterd/ gets wiped off which will also delete glusterd.info
file, isnt that true? Honestly with this data, its not possible to analyze
the issue. You'd need to give us the glusterd log from all the nodes to
progress on it further.


>
>
> Regards
> Abhishek Paliwal
>




-- 
--Atin
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [puzzle] readv operation allocate iobuf twice

[Mohammed Rafi K C]

On 07/15/2016 12:44 AM, Zhengping Zhou wrote:
> Could we just not support user's preset rsp buffer for function
> rpc_clnt_submit, which means remove some relevant parameters of
> function rpc_clnt_submit such like rsphdr, rsphdr_count, rsp_payload,
> rsp_payload_count, rsp_iobref.
>
> The reasons as follow:
>
> 1.We not only should put rsp info to saved_frame, but also should
> duplicate iovec rsp.payload. So we should GF_FREE(
> saved_frame.rsp.rsp_payload)  before mem_put(saved_frame) . It seem's
> ugly.
> wind:
>saved_frame->rsp = *rsp (this rsp is a pointer to local parameter);
>saved_frame->rsp.rsp_payload = iov_dup (rsp.rsp_payload,
> rsp.rsp_payload_count);
> unwind:
>GF_FREE (saved_frame->rsp.rsp_payload);
>memput (saved_frame);
I agree with you. With current code base, it doesn't seems to be helping
much.

But we when we use libgfapi, we could implement zero copy read by
changing the iobuf_get in client3_3_readv, so that the user given
address can be passed as rsp buffer, and if we could give the data in
the same memory , that will help to reduce one in memory copy.

The same can also implement by giving the gluster memory to the user as
a response to libgfapi read.

Apart from this, I don't see any benefit for allocating the memory from
client3_3_readv and giving in rsp struct.


Regards
Rafi KC


>
> 2.If we finish the patch as the way mentioned above, and what about
> rsphdr ? According to the note of rpc_clnt_submit, the rsp_hdr and
> rsp_payload both could preset by users. But we even can't get
> saved_frame before we use rsphdr in socket read state machine, because
> we need rsphdr's content to get saved_frame.
>
> 3.I can't figure out the benefit of presetting rsphdr and rsp_payload.
>
> 2016-07-12 12:38 GMT+08:00 Raghavendra Gowdappa :
>>
>> - Original Message -
>>> From: "Zhengping Zhou" 
>>> To: gluster-devel@gluster.org
>>> Sent: Tuesday, July 12, 2016 9:28:01 AM
>>> Subject: [Gluster-devel] [puzzle] readv operation allocate iobuf twice
>>>
>>> Hi all:
>>>
>>> It is a puzzle to me that we  allocate rsp buffers for rspond
>>> content in function client3_3_readv, but these rsp parameters hasn't
>>> ever been saved to struct saved_frame in submit procedure.
>> Good catch :). We were aware of this issue, but the fix wasn't prioritized. 
>> Can you please file a bug on this? If you want to send a fix (which 
>> essentially stores the rsp payload ptr in saved-frame and passes it down 
>> during rpc_clnt_fill_request_info - as part of handling 
>> RPC_TRANSPORT_MAP_XID_REQUEST event in rpc-clnt), please post a patch to 
>> gerrit and I'll accept it. If you don't have bandwidth, one of us can send 
>> out a fix too.
>>
>> Again, thanks for the effort :).
>>
>> regards,
>> Raghavendra
>>
>>> Which means
>>> the iobuf will reallocated by transport layer in function
>>> __socket_read_accepted_successful_reply.
>>> According to  the commnet of fucntion rpc_clnt_submit :
>>> 1. Both @rsp_hdr and @rsp_payload are optional.
>>> 2. The user of rpc_clnt_submit, if wants response hdr and payload in its
>>> own
>>> buffers, then it has to populate @rsphdr and @rsp_payload.
>>> 
>>> The rsp_payload  is optional, ransport layer will not reallocate
>>> rsp buffers if
>>> it populated. But the fact is readv operation will allocate rsp buffer 
>>> twice.
>>>
>>> Thanks
>>> Zhengping
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Duplicati UUID entries

[ABHISHEK PALIWAL]

Hi,

After a long time I am posting one more issue here.

We have two board and glusterfs in sync on both of them, and our test case
to restart one board continuously but in this TestCase we are getting
duplicate entries of UUID in "gluster peer status" command and it is very
rarely seen.


So, I just want to know the possible reason behind this problem.

There is no possibility of deletion of glusterd.info file and I think if
this is not present then only gluster will generate new UUID for the peer.


Regards
Abhishek Paliwal
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel